Commit Graph

24 Commits

Author SHA1 Message Date
Bruce Martin
59f989d26f initial support for corpora schema conventions (#1676)
* initial support for corpora schema conventions

* remove debugging print

* add corpora util module

* tests

* lint

* PR review edits

* PR changes

* more PR changes

* more PR chnages

* PR fixes

* formatting

* PR updates

* lint

* PR review
2020-07-28 17:32:27 -07:00
Ambrose J Carr
27f75641cc update index links to include 'post' directory (#1666)
* update index links to include 'post' directory

* rebuild github pages site

* document how to develop documentation

* build, not serve
2020-07-23 21:49:57 -07:00
Matt Weiden
152c07a28e Change default branch from 'master' to 'main' (#1589)
The default branch has been changed. This PR cleans up references to
`master` and fixes links that would otherwise be broken.

For more background see the following references:
* https://www.independent.co.uk/life-style/gadgets-and-tech/news/github-master-slave-slavery-whitelist-language-inclusive-a9568576.html
* https://tools.ietf.org/id/draft-knodel-terminology-00.html
2020-06-25 14:04:30 -07:00
Timmy Huang
83376627e8 1510-smoke-test (#1548)
* 1510-smoke-test

* config default

* update tests

* update test config

* fix linter errors

* more comments

* address comments

* use npm install in push_tests.yml

* use environment.default.json

* adding docs

* Take care of @mweiden's nits

* Save screenshots in the __tests__/screenshots/ directory

* typo

* docs

* Add chart tests (#1580)

* merge tests

* check if bin creation returned null before rendering charts (#1576)

* check if bin creation returned null before rendering charts

* refactor chart rendering into functions (#1577)

* little fixes from PR

* reintroduce fix to check for null values

* change getAllByClass to return element

* slice instead

* new stackedbar test

* feedback-1573-test (#1579)

* feedback-1573-test

* enable whole test set

* revert tests

Co-authored-by: Timmy Huang <tihuan@users.noreply.github.com>

* tweak test to actually render chart

* include snapshot

* remove async

* fix getAllHistograms

* properly grab id

Co-authored-by: Timmy Huang <tihuan@users.noreply.github.com>

Co-authored-by: Matt Weiden <538456+mweiden@users.noreply.github.com>
Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com>
2020-06-24 11:45:39 -07:00
Matt Weiden
df6b42f5d6 Update client development documentation (#1504)
Update documentation
* on installing the server
* on how to get nodejs hot-reloading to work
* on running standard and annotations smoke tests with hot-reloading
2020-06-04 12:34:46 -07:00
Matt Weiden
a2047b90ae Make the URL hit by jest smoke tests configurable (#1420)
* Make the URL hit by jest smoke tests configurable

* Fix typo

* Make names consistent
2020-04-22 15:23:11 -07:00
Matt Weiden
7d4d360e52 Refactor developer convenience scripts (#1377)
* Cleanup the backend-dev convenience method

* Add the frontend_dev convenience method

frontend_dev is a soup-to-nuts convenience method for setting up the FE
development environment with node running a the client code on port 3000
with the a separate cellxgene package serving the API over port 5005 in
the background.

The script can be run from Finder.

* Update the developer scripts documentation

* Remove the 'test' make target in the client Makefile

Rationale:
* Given how long the smoke tests take to run, it is unlikely that
  developers will want to run all tests together.
* It is unlikely that developers will have set up the backend server
  properly for the tests to pass.
* Available commands should be safe-ish and not lend themselves to
  confusing errors.
* You can still group tests by concatenating them in a make command, as
  in `make unit-test smoke-test`.
* This target isn't used in any of our CI pipelines -- KISS.

* Some version of python3...

* Minor typos in docs

* Respond to feedback from @bkmartinjr

* Make adjustments so that DATASET path is predictable

* Simplify environment defaults a bit
2020-04-15 10:32:19 -07:00
Matt Weiden
14021db239 Clean up main Makefile (#1376)
* Remove unnecessary make target alias

* Remove installing from build directory

* Update documentation
2020-04-09 17:35:49 -07:00
bmccandless
907cc634f5 server refactor (#1140)
This PR contains a refactoring to make adding new features easier.

The new features include supporting the tiledb format, and the multi dataset application.

The refactoring includes

Simplifying the directory structure and files.
a class structure to handle annotations (currently one type: AnnotationsLocalFile).
a class to handle application configuration
a class structure to handle matrix data (currently AnndataAdaptor and CxgAdaptor). CxgAdaptor uses tiledb.
Algorithms that were previously dependent on the scanpy anndata object are now generalized to work with an abstract interface.
The multi dataset option is not fully supported yet, and so the option to use it is hidden.
Use "cli launch --dataroot ..."
To access this feature.

All combinations of app single dataset/ app multi dataset and AnndataAdaptor/CxgAdaptor work with all the features, such as annotations, ontologies, diffexp.
2020-02-19 10:22:35 -08:00
Matt Weiden
d30087ca63 Add frontend smoke tests for annotations (#1154)
* Add smoke test for annotations features

* Do not save during annotations tests

* Fix botched rebase in dev guidelines

* Revert "Do not save during annotations tests"

This reverts commit f0bd970bb2.

* Respond to feedback from @bkmartinjr
2020-02-14 17:23:45 -08:00
Matt Weiden
ac13b31e13 Collect all env vars in one, easy-to-find place (#1149)
* Collect all env vars in one, easy-to-find place

Past state:
* Default environement variables were stored in both client/package.json
and client/__tests__/e2e/config.js
* Constants that should have been linked--like the cellxgene server port
during testing--were repeated.

With this commit:
* All environment variables are parameterized
* All environment variables are packaged in default env files

* Move npm scripts to client Makefile

* Respond to feedback from @seve and @bkmartinjr
2020-02-12 12:50:48 -08:00
Matt Weiden
e1ff800980 Rename Makefile target build-server to build-cli (#1103)
The `build-server` makefile target does not actually build the server
module, rather it builds the CLI. Rename this target to make it less
confusing.
2020-01-09 07:52:20 -08:00
Matt Weiden
0d2cb14796 Fix smoke test instructions (#1087) 2020-01-02 16:32:00 -08:00
Matt Weiden
53a6d01fa3 Notify users of new versions of cellxgene (#1078)
* Notify users of new versions of cellxgene

Fixes https://github.com/chanzuckerberg/cellxgene/issues/683

* Do not use PyGithub client

* Protect against AttributeError

* Document that all version tags must follow SemVer

* Release tags `should -> MUST` follow semantic versioning
2019-12-31 17:43:11 -08:00
Matt Weiden
f3015cb9df Makefile modularity, test targets, and auto-formatting (#1070)
* Fix Makefile whitespace and .PHONY use

* Fix Makefile filename

* Modularize Makefile into client and server Makefiles

Part of the reason that the Makefile in the root directory is a bit
complicated is that it tries to handle tasks that can be handled
separately in the client and server modules.

This commit pushes some of the make logic specific to each module into
their own makefiles and calls out to those makefiles from that in the
project root.

* Add auto-formatting to client and server modules

One thing that can make linting faster is auto-formatting. This commit
adds the yapf auto-formatting tool to the server module and uses
eslint's "fix" functionality to speed up the linting/formatting process.

* Add yapf for automatic code formatting

* Add a root test target that calls sub-tests

* Apply yapf to python files

* Do not duplicate npm commands, simply pass through

* Update documentation

* Do not shadow reserved word len

* Add general test target

* Fix make call in dev-env

* Use black instead of yapf

* Run flake8 from the root directory

* Revert "Apply yapf to python files"

This reverts commit cdca128a01.

* Apply black to python code

* Resolve lint errors resulting from black format

* Add explanation of server unit tests in dev guidelines
2019-12-27 14:43:37 -08:00
Charlotte Weaver
3e61bb807d e2e testing documentation (#1000) 2019-11-03 10:56:28 -08:00
Bruce Martin
60a23b681c QOL documentation: csweaver contributions from PR #912 (#977)
* csweaver changes from PR #912

* fix formatting error
2019-10-24 08:44:36 -07:00
Charlotte Weaver
63af79d3f8 Add developer guidelines (#769)
* Add developer guidelines

* minor formatting

* PR clarifications/lint

* more pr fixes

* link fix

* below->above

* pr suggestions
2019-05-21 13:57:34 -07:00
Charlotte Weaver
86eb01eb2c improve release process (#752)
* Add --no-cache-dir to make release-install target

Prevents installing from cache so you get the freshest release

* Testing releases is not optional

* Updated release documentation
2019-05-08 09:36:00 -07:00
Charlotte Weaver
40ad283107 create server testing doc (#592) 2019-02-13 15:35:45 -08:00
Bruce Martin
2e9525741f doc divergence warning (#591) 2019-02-08 11:00:51 -08:00
Charlotte Weaver
f737cc4ee4 Build improvements (#577) 2019-02-04 14:15:35 -08:00
Bruce Martin
b90447c387 binary wire format with flatbuffers (#509)
* first flatbuffer schema

* do not lint auto-generated files

* add flatbuffers package

* add flatbuffer module

* wire up /data/X/T route

* use flatbuffers for matrix data fetc

* clarity and comments

* add flatbuffer layout route

* clean up obsolete code

* fix tests

* move flake8 config to setup.cfg

* add comments

* lint

* rework layout routes for fbs

* add more type support to fbs

* lint

* add flatbuffer support for annotations

* function name improvements

* fix botched merge with master

* remove unused import

* route cleanup for flatbuffers

* rename function for clarity

* add missing globals to Jest tests

* fix client JS tests

* fix routes for Python tests

* comments for clarity

* non-finite floating point hardening

* more non-finite number handling

* lint

* fix tests for summarizeAnnotations

* harden diffexp calculation against FP errors

* cleanup unused code

* lint

* add encoding tests for flatbuffers

* application type specified as strings

* fix spelling error

* improve variable names

* add note about documentation gap

* rename FBS DataFrame to Matrix
2019-01-09 14:26:05 -08:00
Justin Kiggins
bc59074300 new docs (#525)
* new docs

* moves development docs to dev_docs
2018-12-20 14:28:05 -08:00