Github Actions Workflow updates compatibility matrix:
- Added MacOS Catalina and Big Sur to test compat matrix
- Added Python 3.9 to test compat matrix, but avoid running 3.9 for matrix jobs that do not have `tables` pypi build available for the given env
- Maintains running tests on both cellxgene main branch and latest pypi release.
- Add explicit matrix exclusions for matrix combinations that will never pass (see comments).
- Numerous refactorings to the workflow config to simplify matrix. Basically a rewrite.
- The anndata pkg is now tested at a pinned release and at latest release, but no longer using `master` branch version. To limit cross-product explosion of matrix jobs, the pinned anndata version is only tested on py3.8 and cellxgene latest release.
- Run unit and smoke tests in a single job, to improve speed, reduce workflow complexity and the number of jobs. Also fixes the redundant testing of unit tests. Within each job, the unit and smoke tests are run in separate steps for ease of troubleshooting.
- Fixed termination of backend server to allow both smoke tests to run within a single job (both attempt to use 5005 port, sequentially, but first server was not being terminated).
- Replaced `continue-on-error: true` with `fail-fast: false`, which allows all matrix jobs to run independently, while also ensuring the that entire workflow is flagged as failed if any matrix job fails
- The `smoke-test-annotations` fail intermittently and have been disabled. Fix will be addressed in story: https://app.zenhub.com/workspaces/single-cell-5e2a191dad828d52cc78b028/issues/chanzuckerberg/cellxgene/2433
* Updates due dependency version changes.
h5py recently changes and now values once returned as str are now returned as bytes.
This would have caused a much larger change, so instead the version is restricted to <3.0.0.
This caused the bulk of the testing failues.
A few other changes were needed to make a few other tests pass.
#1959
* add oauth authentication
Add support for OAuth2.
Change the interface to AuthTypeBase
- better handling of config parameters
- add a complete_setup function for additional setup steps
Added a function wrapper to enforce authentication for the
routes that require authenticaiton.
* change fsspec requirement
fsspec 0.8.0 breaks our tests
it imports a module that is does not require.
Support for sparse tiledb arrays for the X matrix
1. cxgtool can now output sparse matrices
2. cxg_adaptor and diffexp_cxg updated to handle sparse matrices
3. added a test in test_diffexp to test sparse diffexp and get_X_array
6.7 does not have the `hidden` flag used in the code. Users building the
app with an older version of click within the current range specified by
requirements.txt may fail.
* hello world
* stuff
* successful build
* updates"
* maybe a basic example
* simplify
* reamde into dockerfile
* some more stuff
* Release procfile
* package.json at top levle
* don't release in procfile for now
* more package.json stuff
* copy assets
* merge master
* not in the relase phase
* revert not necessary
* pin gunicorn version
* reset common.mk
* Update package.json
Co-authored-by: Madison Dunitz <dunitzm@gmail.com>
* s3 region should have a single config param
The s3 region can also now be automatically determined to further
reduce errors.
This patch also fixes a bug with order of handling the config params.
The tiledb config needs to be fixed before attempting to load
(need to handle_adaptor before handle_single_dataset)
* Introduce a config file to cellxgene
The config file format is in yaml. The default config is located
in server/common/default_config.py. A user may create a yaml file
that contains a subset of these fields. It can be used during cellxgene
launch, or for hosted cellxgene.
The code has also been refactored. Much of the logic to check arguments
has moved from launch to app config.
It is now possible to set the tiledb context parameters using the config
file. Other feature will soon be handled in a similar way.
This PR contains a refactoring to make adding new features easier.
The new features include supporting the tiledb format, and the multi dataset application.
The refactoring includes
Simplifying the directory structure and files.
a class structure to handle annotations (currently one type: AnnotationsLocalFile).
a class to handle application configuration
a class structure to handle matrix data (currently AnndataAdaptor and CxgAdaptor). CxgAdaptor uses tiledb.
Algorithms that were previously dependent on the scanpy anndata object are now generalized to work with an abstract interface.
The multi dataset option is not fully supported yet, and so the option to use it is hidden.
Use "cli launch --dataroot ..."
To access this feature.
All combinations of app single dataset/ app multi dataset and AnndataAdaptor/CxgAdaptor work with all the features, such as annotations, ontologies, diffexp.
* revert MatrixProxy; replace with correct use of adata slicing
* work around 0.6 adata slicing bug
* fix incorrect var slice
* simplify slicing of X
* add warning about performance impact of anndata<=0.7
* lint and remove unused code
* improve comment
* lint
* correctly parse versions
* temp files should preserve file suffix if possible - anndata 0.7 compat
* update anndata dependency to 0.6.20
* resolve PR review comments
* add sample ontologies file
* add ontologies reducer
* Move select category to own component
* Dialog and Input factored out
* refactoring categorical, partway
* validationn
* anno
* suggest populates input
* frontend for ontology working
* initial implementation of back-end support for ontologies
* edit is now dialog again
* autosuggest working on edit
* part way through create arbitrary label
* handle choice in function
* pass duplicate cat prop
* editing works
* update test to match new CLI params
* fix occupancy alignment
* edit category as dialogue
* secondary button
* remove stubbed out ontologies
* add label setting upon new label creation
* Update legal characters for labels (#1119)
* Allow any term in the ontology (bypass legal name check)
* Add hyphens and parens to legal characters in names
* improve performance for large ontologies
* correctly handle case where ontologies are disabled
* fix logic error in CLI
Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>
* PR cleanup 1
* lint
* validate user generated labels
* finish hooking up connected suggest component
* protect against undefined callbacks
* Fix illegal characters error message
* break out npm run commands
* fix error detection on label edit
Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>
Co-authored-by: Sidney Bell <sidneymbell@users.noreply.github.com>
* initial cut at backed mode
* make flask multithreading conditional on debug flag
* update X access to support backed mode
* lint
* improve help message for backed mode
* fix tests
* add MatrixProxy to normalize supported matrix types
* add FAQ entry for --backed
* remove use of matrix.T
* clean up
* add ability to disable diffexp from CLI; add hueristic to detect likely slow diffexp calculation, and warn user
* fix tests
* do not print diffexp speed warning if diffexp is disabled
* tweak wording of diffexp speed messages
* add FAQ entry on --disable-diffexp
* revise heuristic for warning about slow diffexp
* use quick tooltip delay on diffexp button
* initial commit of URL support for launch
* lint
* modify tests to use new data locator
* add locator unit tests
* fix typo in faq
* more lint
* update faq per PR review
kaloster