* add sample ontologies file
* add ontologies reducer
* Move select category to own component
* Dialog and Input factored out
* refactoring categorical, partway
* validationn
* anno
* suggest populates input
* frontend for ontology working
* initial implementation of back-end support for ontologies
* edit is now dialog again
* autosuggest working on edit
* part way through create arbitrary label
* handle choice in function
* pass duplicate cat prop
* editing works
* update test to match new CLI params
* fix occupancy alignment
* edit category as dialogue
* secondary button
* remove stubbed out ontologies
* add label setting upon new label creation
* Update legal characters for labels (#1119)
* Allow any term in the ontology (bypass legal name check)
* Add hyphens and parens to legal characters in names
* improve performance for large ontologies
* correctly handle case where ontologies are disabled
* fix logic error in CLI
Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>
* PR cleanup 1
* lint
* validate user generated labels
* finish hooking up connected suggest component
* protect against undefined callbacks
* Fix illegal characters error message
* break out npm run commands
* fix error detection on label edit
Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>
Co-authored-by: Sidney Bell <sidneymbell@users.noreply.github.com>
* Fix Makefile whitespace and .PHONY use
* Fix Makefile filename
* Modularize Makefile into client and server Makefiles
Part of the reason that the Makefile in the root directory is a bit
complicated is that it tries to handle tasks that can be handled
separately in the client and server modules.
This commit pushes some of the make logic specific to each module into
their own makefiles and calls out to those makefiles from that in the
project root.
* Add auto-formatting to client and server modules
One thing that can make linting faster is auto-formatting. This commit
adds the yapf auto-formatting tool to the server module and uses
eslint's "fix" functionality to speed up the linting/formatting process.
* Add yapf for automatic code formatting
* Add a root test target that calls sub-tests
* Apply yapf to python files
* Do not duplicate npm commands, simply pass through
* Update documentation
* Do not shadow reserved word len
* Add general test target
* Fix make call in dev-env
* Use black instead of yapf
* Run flake8 from the root directory
* Revert "Apply yapf to python files"
This reverts commit cdca128a01.
* Apply black to python code
* Resolve lint errors resulting from black format
* Add explanation of server unit tests in dev guidelines
* rename config param label-file
* annotations rework - CLI params, file naming and backups
* lint
* improve cli option error checks
* enable session cookies
* enable session cookies
* add session id
* name annotations file in multi-dataset and multi-user safe manner
* pass data user hash to front-end
* add annotation collection name support to front-end
* add constant for annotation data collection name
* parameterize annotation collection name; make it sticky in the session
* clarify comments
* hard wire a temporary data collection name for testing
* prettier
* test comment
* package command
* set annotations filename dialog
* name and hash are visible
* wire up data collection capture
* add last mod time lookup to data locator
* store data locator in Driver
* save metadata header in labels csv
* lint
* fix tests
* change datetime format to second precision
* initial cut at backed mode
* make flask multithreading conditional on debug flag
* update X access to support backed mode
* lint
* improve help message for backed mode
* fix tests
* add MatrixProxy to normalize supported matrix types
* add FAQ entry for --backed
* remove use of matrix.T
* clean up
* add ability to disable diffexp from CLI; add hueristic to detect likely slow diffexp calculation, and warn user
* fix tests
* do not print diffexp speed warning if diffexp is disabled
* tweak wording of diffexp speed messages
* add FAQ entry on --disable-diffexp
* revise heuristic for warning about slow diffexp
* use quick tooltip delay on diffexp button
* initial commit of URL support for launch
* lint
* modify tests to use new data locator
* add locator unit tests
* fix typo in faq
* more lint
* update faq per PR review
* move app creation to function
* create engine without load
* flake 8 fixes
* cleanup original scanpy test
* add default config
* handle missing data
* test data changes
* unify update
* load data isn't static anymore
* make app a class
* dead code and route removal
* more dead code cleanup
* fix scanpy_engine tests
* lint
* add missing catch in filter parsing
* update scanpy NaN tests
* more fbs tests and dead test removal
* remove forced default for content type negotiation
* bit of cleanup
* more fbs test cleanup
* lint
* remove swagger
* swagger cleanup
* lint
* correctly handle lack of templates
* more dead code removal
* remove unused files
* fix dev build
* lint
* first flatbuffer schema
* do not lint auto-generated files
* add flatbuffers package
* add flatbuffer module
* wire up /data/X/T route
* use flatbuffers for matrix data fetc
* clarity and comments
* add flatbuffer layout route
* clean up obsolete code
* fix tests
* move flake8 config to setup.cfg
* add comments
* lint
* rework layout routes for fbs
* add more type support to fbs
* lint
* add flatbuffer support for annotations
* function name improvements
* fix botched merge with master
* remove unused import
* route cleanup for flatbuffers
* rename function for clarity
* add missing globals to Jest tests
* fix client JS tests
* fix routes for Python tests
* comments for clarity
* non-finite floating point hardening
* more non-finite number handling
* lint
* fix tests for summarizeAnnotations
* harden diffexp calculation against FP errors
* cleanup unused code
* lint
* add encoding tests for flatbuffers
* application type specified as strings
* fix spelling error
* improve variable names
* add note about documentation gap
* rename FBS DataFrame to Matrix
* Scanpy engine now required
Without the --engine param we need to error if scanpy engine cannot be imported rather than waiting for all engines
* CLI options and help matches proposal
(but not all options hooked up yet)
* Flesh out top level args
* Move computation args to engine
* CLI input file (#374)
* Fix test command
(tests still won't work)
* Input is file instead of directory
- also renamed example file
* Csweaver/debug (#376)
* Respect debug flag for logging flask calls
* Add loading messages
* max categories (#377)
* Add max categories
* Rename max_categories to category_selection_limit
* ensure whole numbers
* Add empty filter case
* Filtering dataframes moved to engine instead of rest
* minor changes from PR review
* Minor fixes from PR review
Pass {} instead of none if no filter
chain exceptions
typos
* Use HTTPStatus for all responses
More informative than just the code as an int
* Fill out REST error handling
* Test error routes
* Use HTTP Status for tests too
* factor mime type request into function
* Better mimetype errors
* 404 -> 400 error for bad key
Move to new REST v0.2 communication between front and back-end. This is a first cut implementation which is functional, but will need follow-up enhancements for performance, error checking, etc. Protocol spec is in docs directory.
* Add filtering via indexing
* Using new filter specs
Indexing working
* Added filtering by annotation value
* factor out common methods
* Documentation
* create enum for axis (obs/var)
* Better description for filter's return
* Add boolean to enumerated types
* Augmented enum for scanpy axis
* Create schema for annotations
Based on datatype within scanpy/anndata
+ tests
* remove obsolete schema parse script
* Update rest api to remove old routes and add schema route
* Separate development requirements
* Warning for unsupported datatypes
* include -r requirements.txt in dev
* Merged downcast warnings
* Fixed bug where names were NaNs
Needed to include the index too when creating the series
* Add config endpoint
* Generate app features from CLI selections
* Move features to driver
* Add tests for schema
* Clearer version wording
* python3 version of super
* version from engine to package level
* move features to driver
* Revise layout function to match the new spec
* GET for layout/obs
* PUT Layout (#211)
* PUT Layout
* Csweaver/annotations (#212)
* Update scanpy engine to support the rest v0.2 annotation requests
* GET endpoint for obs annotations + tests
* Documentation
* Test annotations in scanpy engine
* Description for annotation-keys param
* annotation->annotations
* clarified return for annotations
* Use URL query list for annotations fields
* parse_filter parses v0.2 GET filters (#215)
* parse_filter parses v0.2 GET filters
* Don't allow index filters from query params
* Better variable conversion
* Parse filter improvements
- uses default dict
- renamed filter -> query_filter
* Cleanup Tasks (#216)
* Add test_api back into travis build
* Do custom JSON encoding the correct way
* Run cellxgene server in test setup
* Cleanup new tests too
* Option to bind to all interfaces (#225)
app.run("0.0.0.0") instead of app.run("127.0.0.1") binds to all interfaces.
Note: There are comments on the internet that says that the flask server is not up to the task of production serving. I don't think that such scalability concerns apply here, but I was able to get cellxgene working with twistd relatively easily, and we could switch to that if there are scalability concerns.
Test plan: browsed to <ip>:5005/api/v0.2/config on a different host.
* Add filtering via indexing
* Using new filter specs
Indexing working
* Added filtering by annotation value
* factor out common methods
* Documentation
* create enum for axis (obs/var)
* Better description for filter's return
* Add boolean to enumerated types
* Augmented enum for scanpy axis
* Create schema for annotations
Based on datatype within scanpy/anndata
+ tests
* remove obsolete schema parse script
* Update rest api to remove old routes and add schema route
* Separate development requirements
* Warning for unsupported datatypes
* include -r requirements.txt in dev
* Merged downcast warnings
* Fixed bug where names were NaNs
Needed to include the index too when creating the series
* Add config endpoint
* Generate app features from CLI selections
* Move features to driver
* Add tests for schema
* Clearer version wording
* python3 version of super
* version from engine to package level
* move features to driver
* Revise layout function to match the new spec
* GET for layout/obs
* PUT Layout (#211)
* PUT Layout
* Csweaver/annotations (#212)
* Update scanpy engine to support the rest v0.2 annotation requests
* GET endpoint for obs annotations + tests
* Documentation
* Test annotations in scanpy engine
* Description for annotation-keys param
* annotation->annotations
* clarified return for annotations
* Use URL query list for annotations fields
* parse_filter parses v0.2 GET filters (#215)
* parse_filter parses v0.2 GET filters
* Don't allow index filters from query params
* Better variable conversion
* Parse filter improvements
- uses default dict
- renamed filter -> query_filter
* Cleanup Tasks (#216)
* Add test_api back into travis build
* Do custom JSON encoding the correct way
* Run cellxgene server in test setup
* Cleanup new tests too
* Option to bind to all interfaces (#225)
app.run("0.0.0.0") instead of app.run("127.0.0.1") binds to all interfaces.
Note: There are comments on the internet that says that the flask server is not up to the task of production serving. I don't think that such scalability concerns apply here, but I was able to get cellxgene working with twistd relatively easily, and we could switch to that if there are scalability concerns.
Test plan: browsed to <ip>:5005/api/v0.2/config on a different host.
* Fix merge errors
- import warnings was improperly deleted
- scanpy engine tests were totally wrong
* Fix merge error with driver
* PUT /annotations (#235)
* Add query param for annotation name
* fix descriptions, eliminate else clause
* first cut at initial data load on rest 0.2 api
* Annotation var (#248)
* Fix bug strings are always objects in pandas
* Add axis to annotation method
* Add /annotation/var to REST api
* Csweaver/expressiondata (#242)
* Refactor expression method for REST v2
* Add message to QueryStringError
* Fix range filters
* Add GET route for /data
* /data PUT route
* rename expression to data_frame
* clarification of error
* Improve accept type handling
* support all schema types for 0.2 REST API
* remove REST 0.1 code; connect var annotations loading
* config reducer; use config to set data set title; remove obsolete templating code for data set title
* REST 0.2 expression conversion support
* partial port of expression to REST 0.2
* diffexp (#273)
* Add diffexp method to scanpy
and test
* Minor tweaks to diffexp
Get a minimal working version to unblock FE development
* Fixing things git deleted
* cleanup print statements
* Add index test
* additional, partial REST 0.2 bring up of diffexp
* Ignore unstructured annotations for data (#275)
This is a temp hack, need to figure out how to include data.uns if there is only one gene
* diffexp REST 0.2 port finish
* ignore unstructured annotaitons on all routes except layout
* correctly use varDataCache; maintain state during world rebuild
* correct varDataCache use
* temporarily disable all memoization
* refinements to expression data caching
* clear cell sets upon regraph/reset
* update version of REST to 0.2
* Travis build fixes
- comment out cache import
- fix duplicate test name
* Remove dependency from travis
* clarify semantics of config variables
* move generic action helpers into util
