* embedding
* menu bottom left
* button
* change gutters to support lower toolbar
* fix scatterplot layout
* fix tests to match new layout
* fix smoke tests to match new layout
* better sentence, dataset.nObs to top
* scatterplot position
Co-authored-by: bkmartinjr <bruce@chanzuckerberg.com>
* refactor categorical controls state
* lint
* fix race condition in tests
* fix typo
* add missing update on subset
* remove obsolete code
* update jest and puppeteer major version; update all minors
* update when label changes
* remove lint from tests; increase timeouts in e2e tests
* initial refactoring to new async annomatrix
* refine error handling
* fix bad merge
* add continuous legend
* lint
* fix memoization in color table creators
* partial implementation of user defined annotations
* add new annotations action creator file
* first pass at user annotations
* additional user annotation bug fixes
* user annotation auto-save
* unit test cleanup
* lint
* refactor into multiple files
* cleanup
* add column GC
* fix several bugs in user annotations
* remove debug code
* no anonymous functions
* undo redo cleanup
* file cleanup
* scatterplot
* performance
* cleanup
* remove old code
* render in parallel with load
* fix race condition
* simply graph rendering
* render throttle DRY
* fix category label order
* fix typo in e2e test setup
* re-fix the e2e test setup
* be more tolerant of races
* anno matrix unit tests
* temp disable reembedding
* pilot port continuous histo to react-async
* name change
* lint
* fix repaint bug
* typo fix
* update snap to match new ids
* world/universe name cleanup
* move annoMatrix to src dir
* use private underscore naming convention
* fix corner case in all selected
* name cleanup
* add layout control
* init edge case
* lint
* port scatterplot
* fix label indexing bug and improve tests
* port category to react-async
* fix user annotation labelling while subset
* select all of prev layout on layout switch
* fix race with crossfilter update
* prettier lint
* fix misleading comment
* fix url composition in loader
* first pass at crossfilter tests
* lint
* lint
* fix typo
* improved error handling for network errors
* fix memoization bug
* add memo
* refactor for performnce
* add missing single-value handling in select exact parser
* small bugs discovered by tests
* lint
* additional crossfilter unit tests
* remove extraneous comment
* add support for automatic category determination
* lint
* fix render bug in category
* take advantage of schema categories guarantee
* lint
* do not clear history when resetting
* enhanced annomatrix gc
* lint
* finish renaming to follow conventions; fix clone race bug
* lint
* add priority based loading to improve initial data load UX
* crossfilter cache perf
* perf tuning
* remove timers
* documentation
* PR review changes
* PR review changes
* more PR review edits
* improve clarity of comment
* more PR review fixes
* port centroidLabels to use react-async
* remove dead code
* pr review updates
* oops, remove logging
* 1510-smoke-test
* config default
* update tests
* update test config
* fix linter errors
* more comments
* address comments
* use npm install in push_tests.yml
* use environment.default.json
* adding docs
* Take care of @mweiden's nits
* Save screenshots in the __tests__/screenshots/ directory
* typo
* docs
* Add chart tests (#1580)
* merge tests
* check if bin creation returned null before rendering charts (#1576)
* check if bin creation returned null before rendering charts
* refactor chart rendering into functions (#1577)
* little fixes from PR
* reintroduce fix to check for null values
* change getAllByClass to return element
* slice instead
* new stackedbar test
* feedback-1573-test (#1579)
* feedback-1573-test
* enable whole test set
* revert tests
Co-authored-by: Timmy Huang <tihuan@users.noreply.github.com>
* tweak test to actually render chart
* include snapshot
* remove async
* fix getAllHistograms
* properly grab id
Co-authored-by: Timmy Huang <tihuan@users.noreply.github.com>
Co-authored-by: Matt Weiden <538456+mweiden@users.noreply.github.com>
Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com>
* run eslint --fix
* camelcase
* camelCase config part 1
* part 2
* part 3 - removing subscripts
* fix "class-methods-use-this"
* fix "class-methods-use-this"
* fix eslint ignores
* add eslint ignore for set state in update
* reformat comments to appease eslint
* add a11y features
* sort-comp fix
* a11y fix
* add ignore for set state in update
* add a11y htmlFor
* remove unused toast
* remove unnecessary bind
* add ignore for set state in update
* add rel="noopener noreferrer"
Using target="_blank" without rel="noopener noreferrer" is a security risk: see https://mathiasbynens.github.io/rel-noopener
* use arrow function to bind
* remove unused definitions/declarations
* prettier
* remove unused state
* add comments to empty catch blocks remove curly brackets
* escape '
* use eqeqeq
* switch from default export
* remove ignore log
* remove static
* fix import
* revert subscripting config
* clean-up
* remove unnecessary subscript
* fix new errors from master
* change category click handler to a class property
* fix camelcase changes that slipped by
* unused import
* Fix newly introduced ESLint errors from addGenes
* Refactor CSS layout and react logic for layout
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1022
* Menubar should wrap inside middle pane instead of overlapping left
sidebar when window is scrunched
* cellxgene should have a minimum width of 1240px
1. Replace absolute positioning and dimension calculation with css grid
2. Use flexbox for wrapping menubar buttons
* Middle pane (graph) can calculate its own size
* Removing components calculating their size/position relative to
eachother increases modularity, decreases use of global variables
* Improved some scrollbar behavior
* Removed responsive reducer, propagating window size to components
triggers unnecessary events and encourages breaking modularity; doing
this made some components state agnostic
Reference: https://css-tricks.com/snippets/css/complete-guide-grid/
* Reposition the continuous legend
* Small fixes
* Respond to feedback from @colinmegill
* Respond to feedback from @colinmegill
Add more documentation on the renderGraph method.
* refactor reducer to no longer support hover state and hold many labels
* refactor to generate centroidCoordinates for all values of a category
* create hash for function and memoize export
* create button to display all labels for a category
* clear state
* create label for each thing
* calculate on each value
* change to in place modification of map
* switch to for loop with iterator instead of forEach
* use map from centroidLabel instead of creating copy
* adapt for map
* utilize tarrays
* begin documentation
* disable centroids if in zoom mode
* clean up
* persist uncalc coordinates
* document
* clean up and document
* cleanup and document
* fix
* fix undefined labels and document changes
* fix first element skip
* fix conditional recalc
* rename centroidLabel -> centroidLabels
* break out dilation on hover to new reducer
* numerous styling changes for readability
* change centroid icon
* remove colorAccessor from parameters
* recalc centroids on world change
* make label toggle undoable
* remove unused import
* highlight labels on hover
* remove special characters from svg id
* lighten backdrop
* only generate new centroids if they pre-exist
* fix issue with spaces in catagorical value name
* add label buttons to menubar
* change reducer to use colorAccessor and have single toggle
* fix check to see if svg should be rendered
* move svg overlays onto a single svg layer
* dilate on label hover
* remove logs
* allow centroid to update along side regl renders
* allow actions to pass through svg if in zoom mode
* remove artifact from circle
* remove comment
* remove disabling of centroid button
* fix conditional map to screen
* make styling label conditions stricter
* prettier
* refactor onto master
* refactor computePointFlags() to use pointDilation store
* notify when viewport changes
* move svg attributes out of lasso setup and prevent rerenders/writes
* begin playing with transform matrix
* first solution for camera interaction
* create transform using nested groups
* semi-working method using nested groups with transforms
* inversely scale text
* properly do final transform
* cleanup dead / test code
* reinstate original functionality
* breakout centroid labels labels into separate component
* default toggle on for testing
* separate lasso and centroid layers
* remove unnecessary attributes, working hover
* dilation on label hover
* fix dilation on scatterplot
* add dilation on label hover
* break overlay into separate component
* make overlay agnostic to children
* move label mouse actions to centroidlabels component, add overlay state
* remove lasso on switch to camera
* disallow user selection
* fix reducer
* fix subset with continuous color error
* reset labels on color by continuous
* revert centroids on by default
* refactor for nested restructuring
* remove update checking
* remove unused method
* readd deleted hover delay
* remove old centroid setup
* remove centroid from undoable
* cleanup dead code
* remove dead code
* rollback unnecessary changes
* begin adding annotation functionality
* add annotation functionality
* add reset and undo functionality
* change centroids on layout change
* don't create label for unassigned
* add comment pointing out POI for performance
* touch up matrix transform comment
* add comment explaining coordinate space and children's assumed space
* remove dead code
* switch to pure component
* connect centroidLabels to redux
* clean up camera check and null result
* tool tip change
* rename centroid toggle and the like
* fix the misalignment of buttons, also make blueprint use consistent
* fix comment spelling mistakes
* introduce variable for cleaner logic expressions and state assignment
* alter tooltip text to back color by interaction
* remove manual iterator manipulation for forEach()
* remove debounce
* nit fix
* tooltip wording fix
* lint
* freeze objects
* component rendering perf work
* use PureComponent where safe
* remove obsolete WorldUtil code
* make brushable histogram a pure component
* many graph/scatterplot fixes: 722, 882, 758, 886
* fix e2e test breakage caused by graph transform work
* convert webgl point shaders to use flags rather than explicit point size
* remove unused packages from dependencies
* fix typo in regl prop name
* factor common code into util helper file
* legibility tweaks
* menubar 1
* zoom switching
* centering, pixel perfect canvas
* remove dead args and code
* clipping
* remove log
* if
* connect props
* lint
* undo
* logo left, componetize
* graph back to full height
* shadow to top
* do not prematurely call event handlers during render
* change test to deal with async histogram creation
* left section padding
* lint
* adjust graph to account for top bar,
* lasso tests
* refine histogram tests
* remove testing (onlys)
* Add numeric inputs for percentiles
* Define initial values for percentile cutoffs in world reducer
* add percentil to crossfilter dimensions
* worldEqUniverse now handles cloned worlds
* add Dataframe.mapColumns
* Wire up handlers for percentile inputs
* World reducer and stateManager know about continuousPercentileMin/Max
* Create world as universe clone (not pointer) to avoid clobbering vals
* Define basic actions for setting continuousPercentileMin/Max
* Under the hood, deal with percentiles between 0 and 1
* Move percentile inputs to visualization settings menu
* Fix padding for undo/redo buttons
* Trigger world rebuild from percentile actions
* BROKEN - pseudocode for clamping dataframe by percentiles upon world rebuild
* fix error handling on clip quantiles; start world clipping implementation
* more unclipped reorg
* rename crossfilter.percentile to quantile
* simplify schema access
* update continuous legend when scale changes
* update color cache when clip changes
* clip obs annotations and var data when clip quantile changes
* use own fromEntries
* fix tests
* stable non-finite float sort/search
* clarify comments
* fix syntax typo
* use new stand-alone clip
* clip expresssion data
* add select tests for non-finite scalars
* basic styles
* clip UI now requires explicit commit
* reset enable/disable accounts for clip percentiles
* better error messages
* fix bug in undo interaction with programatic min brush selection
* small refactoring
* support clipping of int data
* do not perform unnecessary summarizations
* improve caching of dataframe compiled columns
* add percentile precompute to Dataframe.summarize
* use Dataframe.summarize for clip percentiles
* remove obsolete quantile code from corssfilter
* histogram scale and label Y axis, add unclipped X range labels
* layout tweaks
* scatterplot now updates when clip changes
* improve comments
* remove debugging comment
* rework clip number entry validation for usability
* ui tweaks to histogram colors and layout
* enable undo/redo for clip user action
* refine UI on clip value entry
* api cleanup
* update confusing comment
* clarify purpose of isValidDigitKeyEvent
* fix misleading comment
* apply appropriate button-group classes; do not mix span and div
* variable name and comment changes suggested in PR review
* rename sort to sortArray; remove unused and dead code path
* naming changes suggested in PR review
* code review improvements for clarity
* more small changes from PR review
* lint fixes for PR review
* fix spelling error
* clarify that function performs in-place modification of world
* add comment to clarify intent of range operation
* fix bad indents in comments
* clean up __columnsAccessor comments and code
* improve comments around clipPredicate
* field name consistency
* improve comment on quantiles params
* immutable crossfilter
* first cut at reducer refactor with cascade model
* add initial redo/undo implementation
* small optimization
* integrate expression with history
* add tests for new reducers and fix a couple of small initialization bugs
* treat tiny lasso selections as a clear
* better function name for clarity
* fix undo for differential expression
* remove logging
* fix regression due to bad merge
* cleanup and comments for clarity
* improve undoable configuration for flexibility
* fix stale comments
* remove debugging code from production build
* rename categoricalSelectionState
* rename file
* improve comments
* initial dataframe commit
* initial dataframe port of core app
* rename variables for clarity
* remove unused import
* comment out unused code
* fix array handling bug in crossfilter dimension creation
* allow creation of empty dataframes
* handle non-existent columns
* handle non-existent columns
* revise tests for new dataframe
* comments for clarity
* comments for clarity
* generate bulk add placeholder with real gene names
* fix bug in gene name adding
* more dataframe unit tests
* fix bug - subset from current world, not universe
* put cut and pasted code into a single function
* improve caching of crossfilter
* remove cascading update bug from graph
* more performance work
* improve state handling for scatterplot
* performance optimization of critical path
* add column summarization
* dataframe utils
* add callOnceLazy
* fix tests
* minor updates found during review
* fix misspelling
* remove RESTv02 from function names
* comment cleanup
* cut/icut col parameter defaults to null
* break up large test
* improve tests and comments on dataframe at/has functions
* add Dataframe withCol/dropCol
* expression varData now stored in a dataframe
* dead code cleanup
* use dataframe.summarize()
* test cases for Dataframe.col.summarize
* update test cases for new dataframe summarize
* improve naming
* use new hasCol API
* add comments
* add more Dataframe.withCol tests
* add ability to specify row index in cut operation
* retire subsetVarData function
* correctly handle expression subsetting
* lint and improve comments
* rename cut to subset
* changes based on PR review
* initial dataframe commit
* initial dataframe port of core app
* rename variables for clarity
* remove unused import
* comment out unused code
* fix array handling bug in crossfilter dimension creation
* allow creation of empty dataframes
* handle non-existent columns
* handle non-existent columns
* revise tests for new dataframe
* comments for clarity
* comments for clarity
* generate bulk add placeholder with real gene names
* fix bug in gene name adding
* more dataframe unit tests
* fix bug - subset from current world, not universe
* put cut and pasted code into a single function
* improve caching of crossfilter
* remove cascading update bug from graph
* more performance work
* improve state handling for scatterplot
* performance optimization of critical path
* add column summarization
* dataframe utils
* add callOnceLazy
* fix tests
* minor updates found during review
* fix misspelling
* remove RESTv02 from function names
* comment cleanup
* cut/icut col parameter defaults to null
* break up large test
* improve tests and comments on dataframe at/has functions
* first flatbuffer schema
* do not lint auto-generated files
* add flatbuffers package
* add flatbuffer module
* wire up /data/X/T route
* use flatbuffers for matrix data fetc
* clarity and comments
* add flatbuffer layout route
* clean up obsolete code
* fix tests
* move flake8 config to setup.cfg
* add comments
* lint
* rework layout routes for fbs
* add more type support to fbs
* lint
* add flatbuffer support for annotations
* function name improvements
* fix botched merge with master
* remove unused import
* route cleanup for flatbuffers
* rename function for clarity
* add missing globals to Jest tests
* fix client JS tests
* fix routes for Python tests
* comments for clarity
* non-finite floating point hardening
* more non-finite number handling
* lint
* fix tests for summarizeAnnotations
* harden diffexp calculation against FP errors
* cleanup unused code
* lint
* add encoding tests for flatbuffers
* application type specified as strings
* fix spelling error
* improve variable names
* add note about documentation gap
* rename FBS DataFrame to Matrix
* toast test
* modify webpack build to allow for global CSS; use CSS modules to scope component styling
* remove heatmap
* add label to index resets
* swapping out buttons for blueprint
* adding constants to globals
* swapping out blueprint buttons
* title position, sidebar improvements
* sidebar styles
* adding toasts for validation errors
* zebra section trial
* integrating blueprint, switching buttons over, breaking out scatter
* clean up console
Move to new REST v0.2 communication between front and back-end. This is a first cut implementation which is functional, but will need follow-up enhancements for performance, error checking, etc. Protocol spec is in docs directory.
* Add filtering via indexing
* Using new filter specs
Indexing working
* Added filtering by annotation value
* factor out common methods
* Documentation
* create enum for axis (obs/var)
* Better description for filter's return
* Add boolean to enumerated types
* Augmented enum for scanpy axis
* Create schema for annotations
Based on datatype within scanpy/anndata
+ tests
* remove obsolete schema parse script
* Update rest api to remove old routes and add schema route
* Separate development requirements
* Warning for unsupported datatypes
* include -r requirements.txt in dev
* Merged downcast warnings
* Fixed bug where names were NaNs
Needed to include the index too when creating the series
* Add config endpoint
* Generate app features from CLI selections
* Move features to driver
* Add tests for schema
* Clearer version wording
* python3 version of super
* version from engine to package level
* move features to driver
* Revise layout function to match the new spec
* GET for layout/obs
* PUT Layout (#211)
* PUT Layout
* Csweaver/annotations (#212)
* Update scanpy engine to support the rest v0.2 annotation requests
* GET endpoint for obs annotations + tests
* Documentation
* Test annotations in scanpy engine
* Description for annotation-keys param
* annotation->annotations
* clarified return for annotations
* Use URL query list for annotations fields
* parse_filter parses v0.2 GET filters (#215)
* parse_filter parses v0.2 GET filters
* Don't allow index filters from query params
* Better variable conversion
* Parse filter improvements
- uses default dict
- renamed filter -> query_filter
* Cleanup Tasks (#216)
* Add test_api back into travis build
* Do custom JSON encoding the correct way
* Run cellxgene server in test setup
* Cleanup new tests too
* Option to bind to all interfaces (#225)
app.run("0.0.0.0") instead of app.run("127.0.0.1") binds to all interfaces.
Note: There are comments on the internet that says that the flask server is not up to the task of production serving. I don't think that such scalability concerns apply here, but I was able to get cellxgene working with twistd relatively easily, and we could switch to that if there are scalability concerns.
Test plan: browsed to <ip>:5005/api/v0.2/config on a different host.
* Add filtering via indexing
* Using new filter specs
Indexing working
* Added filtering by annotation value
* factor out common methods
* Documentation
* create enum for axis (obs/var)
* Better description for filter's return
* Add boolean to enumerated types
* Augmented enum for scanpy axis
* Create schema for annotations
Based on datatype within scanpy/anndata
+ tests
* remove obsolete schema parse script
* Update rest api to remove old routes and add schema route
* Separate development requirements
* Warning for unsupported datatypes
* include -r requirements.txt in dev
* Merged downcast warnings
* Fixed bug where names were NaNs
Needed to include the index too when creating the series
* Add config endpoint
* Generate app features from CLI selections
* Move features to driver
* Add tests for schema
* Clearer version wording
* python3 version of super
* version from engine to package level
* move features to driver
* Revise layout function to match the new spec
* GET for layout/obs
* PUT Layout (#211)
* PUT Layout
* Csweaver/annotations (#212)
* Update scanpy engine to support the rest v0.2 annotation requests
* GET endpoint for obs annotations + tests
* Documentation
* Test annotations in scanpy engine
* Description for annotation-keys param
* annotation->annotations
* clarified return for annotations
* Use URL query list for annotations fields
* parse_filter parses v0.2 GET filters (#215)
* parse_filter parses v0.2 GET filters
* Don't allow index filters from query params
* Better variable conversion
* Parse filter improvements
- uses default dict
- renamed filter -> query_filter
* Cleanup Tasks (#216)
* Add test_api back into travis build
* Do custom JSON encoding the correct way
* Run cellxgene server in test setup
* Cleanup new tests too
* Option to bind to all interfaces (#225)
app.run("0.0.0.0") instead of app.run("127.0.0.1") binds to all interfaces.
Note: There are comments on the internet that says that the flask server is not up to the task of production serving. I don't think that such scalability concerns apply here, but I was able to get cellxgene working with twistd relatively easily, and we could switch to that if there are scalability concerns.
Test plan: browsed to <ip>:5005/api/v0.2/config on a different host.
* Fix merge errors
- import warnings was improperly deleted
- scanpy engine tests were totally wrong
* Fix merge error with driver
* PUT /annotations (#235)
* Add query param for annotation name
* fix descriptions, eliminate else clause
* first cut at initial data load on rest 0.2 api
* Annotation var (#248)
* Fix bug strings are always objects in pandas
* Add axis to annotation method
* Add /annotation/var to REST api
* Csweaver/expressiondata (#242)
* Refactor expression method for REST v2
* Add message to QueryStringError
* Fix range filters
* Add GET route for /data
* /data PUT route
* rename expression to data_frame
* clarification of error
* Improve accept type handling
* support all schema types for 0.2 REST API
* remove REST 0.1 code; connect var annotations loading
* config reducer; use config to set data set title; remove obsolete templating code for data set title
* REST 0.2 expression conversion support
* partial port of expression to REST 0.2
* diffexp (#273)
* Add diffexp method to scanpy
and test
* Minor tweaks to diffexp
Get a minimal working version to unblock FE development
* Fixing things git deleted
* cleanup print statements
* Add index test
* additional, partial REST 0.2 bring up of diffexp
* Ignore unstructured annotations for data (#275)
This is a temp hack, need to figure out how to include data.uns if there is only one gene
* diffexp REST 0.2 port finish
* ignore unstructured annotaitons on all routes except layout
* correctly use varDataCache; maintain state during world rebuild
* correct varDataCache use
* temporarily disable all memoization
* refinements to expression data caching
* clear cell sets upon regraph/reset
* update version of REST to 0.2
* Travis build fixes
- comment out cache import
- fix duplicate test name
* Remove dependency from travis
* clarify semantics of config variables
* move generic action helpers into util
* mocks for redux refactor - for discussion
* more API design on redux refactor
* add new reuqired dependencies for build
* change babel target to use modern browser
* remove dead code
* remove dead code - joy plots
* checkpoint on redux refactoring
* checkpoint on redux refactoring
* fix mistaken rebase conflict resolution
* dead code removal; add name to dataframe backmap
* rename dataframe to universe
* update eslint config to more closely match prettier
* lint
* more eslint updates to match prettier
* additional config to make eslint match prettier
* add expression data to Universe/World
* remove obsolete reducers
* lint fixes
* more eslint cleanup
* lint
* lint
* fix but in countAllOnes when dimensions gt 1
* lint; do not display name metadata field
* lint; colors refactor
* lint; colors refactor
* update comments
* first cut at regraph and reset
* enable object-curly-braces consistent mode
* lint, handle regraph with no selection
* fix expression scatterplot bugs
* fix regression legend display
* add expression data cache
* remove console logging
* reset cell color on regraph/reset
* remove obsolete server URLs
* rename UniverseV01 to Universe_REST_API_v01
* add additional comments on the varDataCache
* merge universe reducer into controls reducer; simplify initialization-related actions
* use spread operator
* fix erroneous comment
* convert universe and world state to plain objects, and functionalize supporting code (remove ES6 classes)
* use spread operator
* lint
* improve variable names
* rename obsCrossfilter to crossfilter and obsDimensionMap to dimensionMap
* rename controls2 to controls
Fixes the build error in the old version.
Reverted the API version since we changed directions from updating the API to refactoring the server structure instead.
Bruce Martin