Commit Graph

51 Commits

Author SHA1 Message Date
Charlotte Weaver
eb3b3fc59a Reorganize order of methods in engine and rest (#305) 2018-10-10 16:44:05 -07:00
Charlotte Weaver
11ef5ae51b REST Error handling (#299)
* Use HTTPStatus for all responses

More informative than just the code as an int

* Fill out REST error handling

* Test error routes

* Use HTTP Status for tests too

* factor mime type request into function

* Better mimetype errors

* 404 -> 400 error for bad key
2018-10-08 14:17:56 -07:00
Charlotte Weaver
8be0833d23 /layout put-> post (#301) 2018-10-08 13:38:13 -07:00
Philipp A
8674ad1f7f CLI improvements (#221)
* CLI improvements

* Circumvent apparent bug with subparsers.required
2018-10-04 16:41:14 -07:00
Charlotte Weaver
187bbfdcf7 /data/var (#295)
* Upgrade version of scanpy

* /data/var

This works for everything except the case where there is only one gene. Anndata flattens X when there is only one var thus causing the transpose to fail.

* Fix edge case when an axis (obs/var) only contains 1 element
2018-10-03 15:09:46 -07:00
Bruce Martin
eeec842ad0 Restv2 feature branch merge to master (#284)
Move to new REST v0.2 communication between front and back-end.   This is a first cut implementation which is functional, but will need follow-up enhancements for performance, error checking, etc.    Protocol spec is in docs directory.

* Add filtering via indexing

* Using new filter specs

Indexing working

* Added filtering by annotation value

* factor out common methods

* Documentation

* create enum for axis (obs/var)

* Better description for filter's return

* Add boolean to enumerated types

* Augmented enum for scanpy axis

* Create schema for annotations

Based on datatype within scanpy/anndata
+ tests

* remove obsolete schema parse script

* Update rest api to remove old routes and add schema route

* Separate development requirements

* Warning for unsupported datatypes

* include -r requirements.txt in dev

* Merged downcast warnings

* Fixed bug where names were NaNs

Needed to include the index too when creating the series

* Add config endpoint

* Generate app features from CLI selections

* Move features to driver

* Add tests for schema

* Clearer version wording

* python3 version of super

* version from engine to package level

* move features to driver

* Revise layout function to match the new spec

* GET for layout/obs

* PUT Layout (#211)

* PUT Layout

* Csweaver/annotations (#212)


* Update scanpy engine to support the rest v0.2 annotation requests

* GET endpoint for obs annotations + tests

* Documentation

* Test annotations in scanpy engine

* Description for annotation-keys param

* annotation->annotations

* clarified return for annotations

* Use URL query list for annotations fields

* parse_filter parses v0.2 GET filters (#215)

* parse_filter parses v0.2 GET filters

* Don't allow index filters from query params

* Better variable conversion

* Parse filter improvements

- uses default dict
- renamed filter -> query_filter

* Cleanup Tasks (#216)

* Add test_api back into travis build

* Do custom JSON encoding the correct way

* Run cellxgene server in test setup

* Cleanup new tests too

* Option to bind to all interfaces (#225)

app.run("0.0.0.0") instead of app.run("127.0.0.1") binds to all interfaces.

Note: There are comments on the internet that says that the flask server is not up to the task of production serving.  I don't think that such scalability concerns apply here, but I was able to get cellxgene working with twistd relatively easily, and we could switch to that if there are scalability concerns.

Test plan: browsed to <ip>:5005/api/v0.2/config on a different host.

* Add filtering via indexing

* Using new filter specs

Indexing working

* Added filtering by annotation value

* factor out common methods

* Documentation

* create enum for axis (obs/var)

* Better description for filter's return

* Add boolean to enumerated types

* Augmented enum for scanpy axis

* Create schema for annotations

Based on datatype within scanpy/anndata
+ tests

* remove obsolete schema parse script

* Update rest api to remove old routes and add schema route

* Separate development requirements

* Warning for unsupported datatypes

* include -r requirements.txt in dev

* Merged downcast warnings

* Fixed bug where names were NaNs

Needed to include the index too when creating the series

* Add config endpoint

* Generate app features from CLI selections

* Move features to driver

* Add tests for schema

* Clearer version wording

* python3 version of super

* version from engine to package level

* move features to driver

* Revise layout function to match the new spec

* GET for layout/obs

* PUT Layout (#211)

* PUT Layout

* Csweaver/annotations (#212)


* Update scanpy engine to support the rest v0.2 annotation requests

* GET endpoint for obs annotations + tests

* Documentation

* Test annotations in scanpy engine

* Description for annotation-keys param

* annotation->annotations

* clarified return for annotations

* Use URL query list for annotations fields

* parse_filter parses v0.2 GET filters (#215)

* parse_filter parses v0.2 GET filters

* Don't allow index filters from query params

* Better variable conversion

* Parse filter improvements

- uses default dict
- renamed filter -> query_filter

* Cleanup Tasks (#216)

* Add test_api back into travis build

* Do custom JSON encoding the correct way

* Run cellxgene server in test setup

* Cleanup new tests too

* Option to bind to all interfaces (#225)

app.run("0.0.0.0") instead of app.run("127.0.0.1") binds to all interfaces.

Note: There are comments on the internet that says that the flask server is not up to the task of production serving.  I don't think that such scalability concerns apply here, but I was able to get cellxgene working with twistd relatively easily, and we could switch to that if there are scalability concerns.

Test plan: browsed to <ip>:5005/api/v0.2/config on a different host.

* Fix merge errors

- import warnings was improperly deleted
- scanpy engine tests were totally wrong

* Fix merge error with driver

* PUT /annotations (#235)

* Add query param for annotation name

* fix descriptions, eliminate else clause

* first cut at initial data load on rest 0.2 api

* Annotation var (#248)

* Fix bug strings are always objects in pandas

* Add axis to annotation method

* Add /annotation/var to REST api

* Csweaver/expressiondata (#242)

* Refactor expression method for REST v2

* Add message to QueryStringError

* Fix range filters

* Add GET route for /data

* /data PUT route

* rename expression to data_frame

* clarification of error

* Improve accept type handling

* support all schema types for 0.2 REST API

* remove REST 0.1 code; connect var annotations loading

* config reducer; use config to set data set title; remove obsolete templating code for data set title

* REST 0.2 expression conversion support

* partial port of expression to REST 0.2

*  diffexp (#273)

* Add diffexp method to scanpy

and test

* Minor tweaks to diffexp

Get a minimal working version to unblock FE development

* Fixing things git deleted

* cleanup print statements

* Add index test

* additional, partial REST 0.2 bring up of diffexp

* Ignore unstructured annotations for data (#275)

This is a temp hack, need to figure out how to include data.uns if there is only one gene

* diffexp REST 0.2 port finish

* ignore unstructured annotaitons on all routes except layout

* correctly use varDataCache; maintain state during world rebuild

* correct varDataCache use

* temporarily disable all memoization

* refinements to expression data caching

* clear cell sets upon regraph/reset

* update version of REST to 0.2

* Travis build fixes

- comment out cache import
- fix duplicate test name

* Remove dependency from travis

* clarify semantics of config variables

* move generic action helpers into util
2018-10-01 14:58:46 -07:00
Charlotte Weaver
39414503bd Revert "Format loaded dataset" 2018-08-14 10:40:21 -07:00
Charlotte Weaver
b884487789 Merge branch 'master' into csweaver/api-v2-init 2018-08-13 15:48:06 -07:00
Charlotte Weaver
ddaa7016a5 Quote formatting
' => "
2018-08-13 15:00:23 -07:00
Charlotte Weaver
d44d267bda Initial tests for the rest v2 refactor
testing the annotations and the data format validation
2018-08-10 16:57:41 -07:00
Charlotte Weaver
55fa9b892a Splitting the format validation and mandatory annotations 2018-08-10 16:38:07 -07:00
Charlotte Weaver
21a1f00a64 Set static folder to correct location 2018-08-09 14:25:11 -07:00
Charlotte Weaver
e96653b826 Serving favicon correctly 2018-08-09 14:14:45 -07:00
Charlotte Weaver
58000c1815 Format loaded dataset
- create cell and gene ids
- recast numbers to float32/int32
2018-08-09 11:23:43 -07:00
Charlotte Weaver
246f4e41f0 Merge pull request #156 from chanzuckerberg/csweaver/infer-metadata
Infer schema if none exists
2018-08-08 11:56:13 -07:00
Charlotte Weaver
af85792cce Don't forget unsigned ints! 2018-08-08 11:20:01 -07:00
Charlotte Weaver
f4780a9806 Moved add to parser out of try block. 2018-08-08 11:18:06 -07:00
Charlotte Weaver
2a11af2550 Formatting 2018-08-07 17:01:19 -07:00
Charlotte Weaver
e5c7c0c44a Warn instead if print when scanpy isn't available 2018-08-07 16:57:07 -07:00
Charlotte Weaver
bf19bb7dce Formatting 2018-08-07 16:38:25 -07:00
Charlotte Weaver
c2b3f331cf Add layout and diffexp calculations to cli options 2018-08-07 16:37:47 -07:00
Charlotte Weaver
499b9551f1 moved scanpy parser to scanpy class 2018-08-07 16:37:02 -07:00
Charlotte Weaver
ccee3e0a38 api host 0.0.0.0 -> 127.0.0.1 2018-08-07 15:43:13 -07:00
Charlotte Weaver
a2b699b05d Infer schema if none exists 2018-08-06 16:39:18 -07:00
Charlotte Weaver
e682028b8e Use flask's current_app 2018-08-03 10:54:43 -07:00
Charlotte Weaver
8dbb131afd Move data to a property of the app object 2018-08-02 16:18:17 -07:00
Charlotte Weaver
e11b905836 Cleaning up import statements 2018-08-02 11:34:00 -07:00
Charlotte Weaver
116cef551c Switch to default timeout for cache 2018-08-02 11:28:44 -07:00
Charlotte Weaver
0a36fac142 10 day cache (down from 1000)
I can expect a scientist to leave this running over the weekend on their laptop and still expect fast results. If someone leaves it running for a few years the data can probably be safely recalculated.
2018-08-01 15:48:56 -07:00
Charlotte Weaver
69fca7ea8b Using flask_caching instead
More recent module, same API
2018-08-01 15:46:04 -07:00
Charlotte Weaver
7642251d44 Add simple cache to backend 2018-08-01 13:32:04 -07:00
Charlotte Weaver
de5e0f4ea7 Address PR review issues 2018-07-20 11:43:31 -07:00
Charlotte Weaver
2c7a11ab80 Added cli command for run 2018-07-19 17:21:28 -07:00
Charlotte Weaver
45f61d87d7 Initial CLI
added basic cli for running
2018-07-18 21:45:07 -07:00
Charlotte Weaver
895cfcebe0 Added initial travis config file 2018-07-18 10:21:24 -07:00
Charlotte Weaver
28d341cf1c Fix default path for dataset 2018-07-17 16:03:06 -07:00
Charlotte Weaver
a5423a6911 Adding minor tests to scanpy calculation code 2018-07-16 11:48:53 -07:00
Charlotte Weaver
05b9a6833d Add tests for scanpy engine 2018-07-16 11:48:53 -07:00
Charlotte Weaver
02799cea8e No relative imports "up" 2018-07-12 12:59:34 -07:00
Charlotte Weaver
ba9e5f1e41 flake8/lint cleanup 2018-07-12 11:52:24 -07:00
Charlotte Weaver
f466ef6553 Added entry point to run cellxgene from console 2018-07-10 12:56:42 -07:00
Charlotte Weaver
0639759a90 Initial packaging working
pip install works if client files are built and moved manually
2018-07-06 15:44:22 -07:00
Charlotte Weaver
cd1b67fd09 Rename diffexpression 2018-07-06 15:41:08 -07:00
Charlotte Weaver
5b9ddfa1a6 Added documentation to filter 2018-07-06 15:40:21 -07:00
Charlotte Weaver
354eacc3c8 Remove cell ids from engine
Previously I just removed them from the driver
2018-07-06 15:38:42 -07:00
Charlotte Weaver
e3bf06ca61 Config file no longer necessary
Config is solely from environment variables instead.
2018-07-06 10:13:24 -07:00
Charlotte Weaver
85506db971 Updating to latest cellxgene client code
Fixes the build error in the old version. 

Reverted the API version since we changed directions from updating the API to refactoring the server structure instead.
2018-07-05 13:37:26 -07:00
Charlotte Weaver
2da62720dd Better documentation for driver/engines
Removed the REST v2.0 documentation in favor of getting v1.0 working with driver/engine model
2018-06-29 17:17:14 -07:00
Charlotte Weaver
56c932a443 Fixed categorical filter not being converted to type, added tests 2018-06-29 17:16:00 -07:00
Charlotte Weaver
40f5b94075 Consistency with quotes
' -> "
2018-06-29 13:14:58 -07:00