Commit Graph
72 Commits
Author SHA1 Message Date
bmccandless 0e48b335be update umap version (#2021)
There is an interface change in 0.5.0 which is not compatible with our
version of scanpy.
2021-01-13 13:01:46 -08:00
bmccandless 7599af252d Update compatibility test (#2009)
Add comment in requirements about h5py and anndata.

  #1963
2020-12-21 14:38:07 -06:00
maniarathi 9a34264f92 Pin tiledb version to exclude 0.7.2 (#1993) 2020-11-30 11:54:33 -05:00
maniarathi 16718f392f Sorry! Undo-ing commit to main 2020-11-24 10:22:24 -05:00
maniarathi ad6b1928e4 Upgrade anndata and h5py 2020-11-24 10:21:24 -05:00
bmccandless b9e132a00c Updates due dependency version changes. (#1960)
* Updates due dependency version changes.

h5py recently changes and now values once returned as str are now returned as bytes.
This would have caused a much larger change, so instead the version is restricted to <3.0.0.

This caused the bulk of the testing failues.
A few other changes were needed to make a few other tests pass.

 #1959
2020-11-01 12:36:38 -08:00
Madison Dunitz 3b6c46ba86 Fix dependency issues in compatibility tests (#1951)
* update reqs

* pin scanpy

* merge in fix for race conditions
2020-10-30 10:47:12 -05:00
Madison Dunitz 946a910ef4 Fix compatibility test (#1948)
* update anndata version and warning about version

* update compatibility tests
2020-10-26 17:23:55 -05:00
maniarathi 9793398737 Add in missing previous crossfilter which was causing the re-embedding feature to fail. (#1936) 2020-10-21 09:16:13 -07:00
maniarathi a5c9ffa880 When reading annotations from tiledb, check if the values are byte literals and if so, decode them. Also pin s3f3 to 0.4.2. (#1788) 2020-08-22 09:42:11 -07:00
bmccandless 905308e09f Move psycopg2==2.7.7 from requirements.txt to requirements-dev.txt (#1747) 2020-08-13 21:20:13 -07:00
maniarathi 0d94c9e092 DRY-ing flatbuffer code (#1716) 2020-08-05 11:56:34 -07:00
Madison Dunitz f632a8db91 Dunitz/db setup (#1619)
* initial database setup
2020-08-03 17:54:06 -05:00
bmccandless 2afa48cf11 add oauth authentication (#1681)
* add oauth authentication

Add support for OAuth2.

Change the interface to AuthTypeBase
  - better handling of config parameters
  - add a complete_setup function for additional setup steps

Added a function wrapper to enforce authentication for the
routes that require authenticaiton.

* change fsspec requirement

fsspec 0.8.0 breaks our tests
it imports a module that is does not require.
2020-07-31 18:16:57 -07:00
bmccandless f3d5081d38 exclude a tiledb version that has a known bug (#1604)
exclude a tiledb version that has a known bug
2020-07-07 08:39:50 -07:00
bmccandless f7585eef1e Support for sparse tiledb arrays for the X matrix (#1496)
Support for sparse tiledb arrays for the X matrix

1. cxgtool can now output sparse matrices
2. cxg_adaptor and diffexp_cxg updated to handle sparse matrices
3. added a test in test_diffexp to test sparse diffexp and get_X_array
2020-05-28 18:36:02 -07:00
bmccandless 4d100d4507 Upgrade to tiledb 2.0 (#1485)
* Upgrade to tiledb 2.0
2020-05-21 14:55:11 -07:00
Matt Weiden e21997799c Upgrade python requirements to click>=7.1.2 (#1472)
6.7 does not have the `hidden` flag used in the code. Users building the
app with an older version of click within the current range specified by
requirements.txt may fail.
2020-05-14 10:04:28 -07:00
Matt Weiden 1ee76826c7 Pin tiledb version 0.5.9 (#1453)
There is a regression in 0.6.0
2020-05-05 10:03:57 -07:00
Eduardo LopezandMadison Dunitz 565a3f148d [feature] Prototype a Heroku deployment of hosted cellxgene (#1427)
* hello world

* stuff

* successful build

* updates"

* maybe a basic example

* simplify

* reamde into dockerfile

* some more stuff

* Release procfile

* package.json at top levle

* don't release in procfile for now

* more package.json stuff

* copy assets

* merge master

* not in the relase phase

* revert not necessary

* pin gunicorn version

* reset common.mk

* Update package.json

Co-authored-by: Madison Dunitz <dunitzm@gmail.com>
2020-04-24 12:44:38 -07:00
bmccandless 03a75ec566 s3 listings were cached, preventing new datasets from being accessible (#1408)
Fixes #1407
2020-04-17 10:14:24 -07:00
bmccandless 5dcfcb5467 s3 region should have a single config param (#1386)
* s3 region should have a single config param

The s3 region can also now be automatically determined to further
reduce errors.

This patch also fixes a bug with order of handling the config params.
The tiledb config needs to be fixed before attempting to load
(need to handle_adaptor before handle_single_dataset)
2020-04-11 15:20:01 -07:00
Bruce Martin ffa95b5a55 update s3fs requirements (#1348) 2020-04-05 09:22:32 -07:00
Bruce Martin 8fac40b6ae various fixes for s3fs use (#1312)
* various fixes for s3fs use

* lint
2020-03-28 22:26:03 -07:00
Bruce Martin 752b9e4ab3 CORS and CSP headers (#1286)
* do in-app compression only for CLI

* CORS and CSP headers

* lint

* add --debug to targets

* lint

* fix botched merge with master
2020-03-24 08:05:37 -07:00
Bruce Martin d196cbc8f6 do in-app compression only for CLI (#1285) 2020-03-24 06:36:40 -07:00
bmccandless 8180be83b8 Introduce a config file to cellxgene (#1264)
* Introduce a config file to cellxgene

The config file format is in yaml.  The default config is located
in server/common/default_config.py.  A user may create a yaml file
that contains a subset of these fields.  It can be used during cellxgene
launch, or for hosted cellxgene.

The code has also been refactored.  Much of the logic to check arguments
has moved from launch to app config.

It is now possible to set the tiledb context parameters using the config
file.  Other feature will soon be handled in a similar way.
2020-03-22 09:34:11 -07:00
Bruce Martin ffdcc24a40 remove pinned tables dependency (#1202) 2020-03-10 07:47:54 -07:00
bmccandless 907cc634f5 server refactor (#1140)
This PR contains a refactoring to make adding new features easier.

The new features include supporting the tiledb format, and the multi dataset application.

The refactoring includes

Simplifying the directory structure and files.
a class structure to handle annotations (currently one type: AnnotationsLocalFile).
a class to handle application configuration
a class structure to handle matrix data (currently AnndataAdaptor and CxgAdaptor). CxgAdaptor uses tiledb.
Algorithms that were previously dependent on the scanpy anndata object are now generalized to work with an abstract interface.
The multi dataset option is not fully supported yet, and so the option to use it is hidden.
Use "cli launch --dataroot ..."
To access this feature.

All combinations of app single dataset/ app multi dataset and AnndataAdaptor/CxgAdaptor work with all the features, such as annotations, ontologies, diffexp.
2020-02-19 10:22:35 -08:00
Bruce Martin 349c413d8b anndata X indexing & version compatibility improvements (#1157)
* revert MatrixProxy; replace with correct use of adata slicing

* work around 0.6 adata slicing bug

* fix incorrect var slice

* simplify slicing of X

* add warning about performance impact of anndata<=0.7

* lint and remove unused code

* improve comment

* lint

* correctly parse versions

* temp files should preserve file suffix if possible - anndata 0.7 compat

* update anndata dependency to 0.6.20

* resolve PR review comments
2020-02-19 09:57:51 -07:00
Sidney Bell a927d733c2 [WIP] Patch for anndata 0.7 (#1132)
* Fix JSON encoding bug introduced by anndata 0.7

* Add anndata csr and csc classes to allowed matrix proxy inputs
2020-01-28 16:12:43 -08:00
d48647a655 Ontologies (#1110)
* add sample ontologies file

* add ontologies reducer

* Move select category to own component

* Dialog and Input factored out

* refactoring categorical, partway

* validationn

* anno

* suggest  populates input

* frontend for ontology working

* initial implementation of back-end support for ontologies

* edit is now dialog again

* autosuggest working on edit

* part way through create arbitrary label

* handle choice in function

* pass duplicate cat  prop

* editing works

* update test to match new CLI params

* fix occupancy alignment

* edit category as dialogue

* secondary button

* remove stubbed out ontologies

* add label setting upon new label creation

* Update legal characters for labels (#1119)

* Allow any term in the ontology (bypass legal name check)

* Add hyphens and parens to legal characters in names

* improve performance for large ontologies

* correctly handle case where ontologies are disabled

* fix logic error in CLI

Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>

* PR cleanup 1

* lint

* validate user generated labels

* finish hooking up connected suggest component

* protect against undefined callbacks

* Fix illegal characters error message

* break out npm run commands

* fix error detection on label edit

Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>
Co-authored-by: Sidney Bell <sidneymbell@users.noreply.github.com>
2020-01-23 17:04:17 -05:00
Bruce Martin 69c639b14d pin anndata to 0.6.22post1 (#1122) 2020-01-22 13:23:05 -08:00
Matt Weiden 327d005a1f Add requests 2.22.0 as a server dependency (#1089) 2020-01-03 10:59:46 -08:00
Bruce Martin 238b7420f7 update deprecated Pandas API (#1021)
* update use of deprecated API

* update pandas depencency
2019-11-08 09:49:52 -08:00
Bruce Martin 711f3b7048 Add support for anndata backed mode (#943)
* initial cut at backed mode

* make flask multithreading conditional on debug flag

* update X access to support backed mode

* lint

* improve help message for backed mode

* fix tests

* add MatrixProxy to normalize supported matrix types

* add FAQ entry for --backed

* remove use of matrix.T

* clean up

* add ability to disable diffexp from CLI; add hueristic to detect likely slow diffexp calculation, and warn user

* fix tests

* do not print diffexp speed warning if diffexp is disabled

* tweak wording of diffexp speed messages

* add FAQ entry on --disable-diffexp

* revise heuristic for warning about slow diffexp

* use quick tooltip delay on diffexp button
2019-10-08 11:16:07 -07:00
Bruce Martin 20317fd08f Add URL data locators to launch sub-command (#920)
* initial commit of URL support for launch

* lint

* modify tests to use new data locator

* add locator unit tests

* fix typo in faq

* more lint

* update faq per PR review
2019-09-15 09:01:53 -07:00
Bruce Martin 84c815cc04 pin h5py to 2.9.0 to temporarily work around regression (#916) 2019-09-11 13:20:15 -07:00
Charlotte Weaver 28c4d28308 extract prepare from main install (#887)
* extract prepare from main install

* add requirements-prepare to manifest
2019-08-21 14:26:01 -07:00
Charlotte Weaver 613f1a911f pin matplotlib dependency (#885)
reordering packages does not work from dist install
2019-08-14 14:21:07 -07:00
Charlotte Weaver 79a503e436 fix bad matplotlib library by installing scanpy first (#881) 2019-08-12 16:42:26 -07:00
Bruce Martin a6142bdf93 improve graph scale and centering (#796)
* add gutter to embedding canvas

* improve layout scale and translate

* fix lint

* pin tables to version 3.5.1

* fix lasso coordinate smoke tests
2019-05-30 13:31:41 -07:00
Bruce Martin a8c2e408d1 update to latest anndata and remove restriction on scipy (#790) 2019-05-24 11:23:35 -07:00
Charlotte Weaver ab4c74a321 remove psutil (#773) 2019-05-18 10:53:45 -07:00
Charlotte Weaver e2ad28a510 exclude recent scipy versions (#770) 2019-05-17 15:13:22 -07:00
Bruce Martin b9a1e30652 large file size guardrails (#763)
* large file guardrails

* fix lint

* PR review

* remove unused import

* use standard slice for CSR

* revert change
2019-05-13 18:13:16 -07:00
Sidney Bell c9a56fa73a Update scanpy version (#688)
Update to version 1.3.7
2019-04-04 12:16:30 -07:00
Charlotte Weaver b6d468376a py37 fixes (#646)
* add python version for nightly build

* update numpy version required

older versions interact poorly on anaconda + python 3.7
2019-03-14 14:51:55 -07:00
Charlotte Weaver d8fc7e40a1 Support python3.7 (#645)
* Support python3.7

* add 3.7 env to travis
2019-03-14 12:08:46 -07:00
Bruce Martin 57c4e9ff33 Flatbuffer cleanup (#598)
* dead code and route removal

* more dead code cleanup

* fix scanpy_engine tests

* lint

* add missing catch in filter parsing

* update scanpy NaN tests

* more fbs tests and dead test removal

* remove forced default for content type negotiation

* bit of cleanup

* more fbs test cleanup

* lint

* remove swagger

* swagger cleanup

* lint

* correctly handle lack of templates

* more dead code removal

* remove unused files

* fix dev build

* lint
2019-02-19 08:50:29 -08:00