* refactor categorical controls state
* lint
* fix race condition in tests
* fix typo
* add missing update on subset
* remove obsolete code
* update jest and puppeteer major version; update all minors
* update when label changes
* remove lint from tests; increase timeouts in e2e tests
* initial refactoring to new async annomatrix
* refine error handling
* fix bad merge
* add continuous legend
* lint
* fix memoization in color table creators
* partial implementation of user defined annotations
* add new annotations action creator file
* first pass at user annotations
* additional user annotation bug fixes
* user annotation auto-save
* unit test cleanup
* lint
* refactor into multiple files
* cleanup
* add column GC
* fix several bugs in user annotations
* remove debug code
* no anonymous functions
* undo redo cleanup
* file cleanup
* scatterplot
* performance
* cleanup
* remove old code
* render in parallel with load
* fix race condition
* simply graph rendering
* render throttle DRY
* fix category label order
* fix typo in e2e test setup
* re-fix the e2e test setup
* be more tolerant of races
* anno matrix unit tests
* temp disable reembedding
* pilot port continuous histo to react-async
* name change
* lint
* fix repaint bug
* typo fix
* update snap to match new ids
* world/universe name cleanup
* move annoMatrix to src dir
* use private underscore naming convention
* fix corner case in all selected
* name cleanup
* add layout control
* init edge case
* lint
* port scatterplot
* fix label indexing bug and improve tests
* port category to react-async
* fix user annotation labelling while subset
* select all of prev layout on layout switch
* fix race with crossfilter update
* prettier lint
* fix misleading comment
* fix url composition in loader
* first pass at crossfilter tests
* lint
* lint
* fix typo
* improved error handling for network errors
* fix memoization bug
* add memo
* refactor for performnce
* add missing single-value handling in select exact parser
* small bugs discovered by tests
* lint
* additional crossfilter unit tests
* remove extraneous comment
* add support for automatic category determination
* lint
* fix render bug in category
* take advantage of schema categories guarantee
* lint
* do not clear history when resetting
* enhanced annomatrix gc
* lint
* finish renaming to follow conventions; fix clone race bug
* lint
* add priority based loading to improve initial data load UX
* crossfilter cache perf
* perf tuning
* remove timers
* documentation
* PR review changes
* PR review changes
* more PR review edits
* improve clarity of comment
* more PR review fixes
* port centroidLabels to use react-async
* remove dead code
* pr review updates
* oops, remove logging
* add case to removing labels
* add disabled state to centroid button
* fix centroid test button clicking order
* don't accidentally un-toggle first color
* run eslint --fix
* camelcase
* camelCase config part 1
* part 2
* part 3 - removing subscripts
* fix "class-methods-use-this"
* fix "class-methods-use-this"
* fix eslint ignores
* add eslint ignore for set state in update
* reformat comments to appease eslint
* add a11y features
* sort-comp fix
* a11y fix
* add ignore for set state in update
* add a11y htmlFor
* remove unused toast
* remove unnecessary bind
* add ignore for set state in update
* add rel="noopener noreferrer"
Using target="_blank" without rel="noopener noreferrer" is a security risk: see https://mathiasbynens.github.io/rel-noopener
* use arrow function to bind
* remove unused definitions/declarations
* prettier
* remove unused state
* add comments to empty catch blocks remove curly brackets
* escape '
* use eqeqeq
* switch from default export
* remove ignore log
* remove static
* fix import
* revert subscripting config
* clean-up
* remove unnecessary subscript
* fix new errors from master
* change category click handler to a class property
* fix camelcase changes that slipped by
* unused import
* Fix newly introduced ESLint errors from addGenes
* Refactor CSS layout and react logic for layout
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1022
* Menubar should wrap inside middle pane instead of overlapping left
sidebar when window is scrunched
* cellxgene should have a minimum width of 1240px
1. Replace absolute positioning and dimension calculation with css grid
2. Use flexbox for wrapping menubar buttons
* Middle pane (graph) can calculate its own size
* Removing components calculating their size/position relative to
eachother increases modularity, decreases use of global variables
* Improved some scrollbar behavior
* Removed responsive reducer, propagating window size to components
triggers unnecessary events and encourages breaking modularity; doing
this made some components state agnostic
Reference: https://css-tricks.com/snippets/css/complete-guide-grid/
* Reposition the continuous legend
* Small fixes
* Respond to feedback from @colinmegill
* Respond to feedback from @colinmegill
Add more documentation on the renderGraph method.
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1433
When selection is deselected the world is reset to the universe the
crossfilter is reset to that in the resetCache, including the embeddings
in the layout_XY dim. However, the embedding selection stays the same.
If the embedding selected is not the default, the embedding shown to the
user will be different than the embedding layout_XY in the crossfilter,
causing lasso selections to be made against the wrong embedding
coordinates
* Add user-defined category-label colors
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1152
As described in https://github.com/chanzuckerberg/cellxgene/issues/1307
* Respond to feedback from @bkmartinjr in nodejs
* Respond to feedback from @bkmartinjr in python
* Add tests to the server module
* Autoformat python, run linter
* Make colors_get error handling specific
* Respond to feedback from @bkmartinjr
* Respond to feedback from @bkmartinjr
* Fix whitespace
* Fix python lint errrors
* Update documentation
* Add --disable-user-colors option to launch and cxgtool.py
* Fix python formatting
* Rename '--disable-user-colors' to '--disable-custom-colors'
* first cut at re-embedding route and back-end support
* update and expand config route tests
* add scanpy_umap
* add reembedding to config route parameters
* front-end support for reembedding fetch and UI
* remove unused imports
* add loading state
* save reembedding in reducer state
* improve withColsFrom
* transmit reembed schema to client; pick unique embedding names
* display embeddings
* format
* lint
* spaces, tab size 2
* lint
* test hack for smoke-test race
* back out hack sleep
* add check for backed mode
* add unit test for reembedding
* lint
* hide re-embedding CLI param from help
* Remove unused variables and imports
* Simplify conditional
* Fix typo
* Do not overprune var data cache
There is a bug in how the universe and world gene sets are constructed
and passed to `ControlsHelpers.pruneVarDataCache` that causes the var
data cache to be over-pruned. This commit fixes the issue.
Consider the following example from the node console:
```
❯ node
Welcome to Node.js v13.5.0.
Type ".help" for more information.
> new Set([1], [2], [3])
Set(1) { 1 }
```
What we really want is the set `Set(3) { 1, 2, 3 }`, which can be
constructed as:
```
> new Set([].concat([1], [2], [3]))
Set(3) { 1, 2, 3 }
```
* Disable ColorBy button for truncated categories
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1156
For categories that have more than 100 labels we truncate the labels in
the UI, but still allowed users to ColorBy these categories. Coloring by
these categories can cause browsers to get bogged down.
This commit disables ColorBy for truncated categories.
* Minor documentation spelling and typo fixes
* Respond to feedback from @liaprins-czi
* Respond to feedback from @colinmegill and @bkmartinjr
* Undo selection appends diffExp genes to user gene list
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1171
Need: When a user performs a differential expression from within
a sub-selection (world) of the data and then resets the selection to all
cells (universe), the differential expression results are no longer
valid.
Approach:
* When the selection is reset, move the top (maxUserDefinedGenes
- len(userDefinedGenes) from the differential expression results to the
list of user defined genes
* Raise maxUserDefinedGenes to 25 to give users more room and
accommodate the extra genes transferred in from differential expression
Other commits:
* Choose different button icons
* Add diff exp genes to user defined genes on subset too
* Respond to feedback from @liaprins-czi and @bkmartinjr
* add simple error message helper
* port all label name pickers to use the new LabelInput component
* use pure components where possible
* cleanup
* more cleanup
* lint
* change new label prompt
* Add undo/redo tests for annotations
Fixes https://github.com/chanzuckerberg/cellxgene/issues/969
... also refactor the tests for DRY.
* Add done()
* Make e2e annotations tests safer to concurrency
* Add data-testclass for save state.
* Simplify tests and make them dependent on save state
* load annotations individually
* fix type check to be more general
* update node CI version from 10 to 12
* node 11
* debug print node version
* travis node version to latest
* try nvm
* remove extraneous node_js statement
* remove node version debugging printf
* incrementally load all annotations and layout
* process annotations and layout as they are loaded
* fix tests
* sort categories incrementally
* incrementally build category view summary; add category loading spinner
* add spinner to continuous metadata
* configure undoable reducer
* incremental crossfilter creation
* improve busy layout
* more layout cleanup
* correctly reconcile categories in schema
* refine layout of lsb spinners
* more spinner layout work
* more spinner layout
* always load layout before obs annotations
* Undo feature: fix case where previous state has no state filter
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1099
When the previous state that the undo feature is trying to roll back to
has no filter state, merging javascript dictionaries result in keeping
the current state filter, preventing the actionFilter from saving the
new state.
* Fix whitespace
* add sample ontologies file
* add ontologies reducer
* Move select category to own component
* Dialog and Input factored out
* refactoring categorical, partway
* validationn
* anno
* suggest populates input
* frontend for ontology working
* initial implementation of back-end support for ontologies
* edit is now dialog again
* autosuggest working on edit
* part way through create arbitrary label
* handle choice in function
* pass duplicate cat prop
* editing works
* update test to match new CLI params
* fix occupancy alignment
* edit category as dialogue
* secondary button
* remove stubbed out ontologies
* add label setting upon new label creation
* Update legal characters for labels (#1119)
* Allow any term in the ontology (bypass legal name check)
* Add hyphens and parens to legal characters in names
* improve performance for large ontologies
* correctly handle case where ontologies are disabled
* fix logic error in CLI
Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>
* PR cleanup 1
* lint
* validate user generated labels
* finish hooking up connected suggest component
* protect against undefined callbacks
* Fix illegal characters error message
* break out npm run commands
* fix error detection on label edit
Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>
Co-authored-by: Sidney Bell <sidneymbell@users.noreply.github.com>
* refactor reducer to no longer support hover state and hold many labels
* refactor to generate centroidCoordinates for all values of a category
* create hash for function and memoize export
* create button to display all labels for a category
* clear state
* create label for each thing
* calculate on each value
* change to in place modification of map
* switch to for loop with iterator instead of forEach
* use map from centroidLabel instead of creating copy
* adapt for map
* utilize tarrays
* begin documentation
* disable centroids if in zoom mode
* clean up
* persist uncalc coordinates
* document
* clean up and document
* cleanup and document
* fix
* fix undefined labels and document changes
* fix first element skip
* fix conditional recalc
* rename centroidLabel -> centroidLabels
* break out dilation on hover to new reducer
* numerous styling changes for readability
* change centroid icon
* remove colorAccessor from parameters
* recalc centroids on world change
* make label toggle undoable
* remove unused import
* highlight labels on hover
* remove special characters from svg id
* lighten backdrop
* only generate new centroids if they pre-exist
* fix issue with spaces in catagorical value name
* add label buttons to menubar
* change reducer to use colorAccessor and have single toggle
* fix check to see if svg should be rendered
* move svg overlays onto a single svg layer
* dilate on label hover
* remove logs
* allow centroid to update along side regl renders
* allow actions to pass through svg if in zoom mode
* remove artifact from circle
* remove comment
* remove disabling of centroid button
* fix conditional map to screen
* make styling label conditions stricter
* prettier
* refactor onto master
* refactor computePointFlags() to use pointDilation store
* notify when viewport changes
* move svg attributes out of lasso setup and prevent rerenders/writes
* begin playing with transform matrix
* first solution for camera interaction
* create transform using nested groups
* semi-working method using nested groups with transforms
* inversely scale text
* properly do final transform
* cleanup dead / test code
* reinstate original functionality
* breakout centroid labels labels into separate component
* default toggle on for testing
* separate lasso and centroid layers
* remove unnecessary attributes, working hover
* dilation on label hover
* fix dilation on scatterplot
* add dilation on label hover
* break overlay into separate component
* make overlay agnostic to children
* move label mouse actions to centroidlabels component, add overlay state
* remove lasso on switch to camera
* disallow user selection
* fix reducer
* fix subset with continuous color error
* reset labels on color by continuous
* revert centroids on by default
* refactor for nested restructuring
* remove update checking
* remove unused method
* readd deleted hover delay
* remove old centroid setup
* remove centroid from undoable
* cleanup dead code
* remove dead code
* rollback unnecessary changes
* begin adding annotation functionality
* add annotation functionality
* add reset and undo functionality
* change centroids on layout change
* don't create label for unassigned
* add comment pointing out POI for performance
* touch up matrix transform comment
* add comment explaining coordinate space and children's assumed space
* remove dead code
* switch to pure component
* connect centroidLabels to redux
* clean up camera check and null result
* tool tip change
* rename centroid toggle and the like
* fix the misalignment of buttons, also make blueprint use consistent
* fix comment spelling mistakes
* introduce variable for cleaner logic expressions and state assignment
* alter tooltip text to back color by interaction
* remove manual iterator manipulation for forEach()
* remove debounce
* nit fix
* tooltip wording fix
* lint
* rename config param label-file
* annotations rework - CLI params, file naming and backups
* lint
* improve cli option error checks
* enable session cookies
* enable session cookies
* add session id
* name annotations file in multi-dataset and multi-user safe manner
* pass data user hash to front-end
* add annotation collection name support to front-end
* add constant for annotation data collection name
* parameterize annotation collection name; make it sticky in the session
* clarify comments
* hard wire a temporary data collection name for testing
* prettier
* test comment
* package command
* set annotations filename dialog
* name and hash are visible
* wire up data collection capture
* cache common results to reduce jank
* add util function in support of annotation labelling
* enable/disable annotation edit menus per issue 972; improve formatting
* PR review requests
* freeze objects
* component rendering perf work
* use PureComponent where safe
* remove obsolete WorldUtil code
* make brushable histogram a pure component
* add action to clear colorMode and colorAccessor if diffexp is removed
* create new colorHelper function
* creater colorHelper for conditionally setting state
* add abbr
* revert abbr
* Connect mouse over events to reducer actions
* Change Styling on hover
* Rename reducer actions to be more descriptive
* Reorder reducer in cascade
* Create centroid calculation util
* Whitespace
* Typo fix, use correct action
* Create centroid calc util
* Create centroid svg setup
* Refactor existing svg layer to toolSVG
* Change calcCentroid signature and centroidXY to match mapPointToScreen
* Add id and styling
* Run prettier
* Set z-index to 999
* Draw the label
* Introduce the centroid SVG, refactor code to allow both SVG layers
* Add text label and compute radius based on population
* Implement optional chaining
* Update font family
* Optimize calcMeanCentroid()
* Create and utilize calcMedianCentroid()
* Remove mass circle from label
* Remove styling change on hover
* Remove reducer action logs
* Prettier
* Swap out binds for arrow functions
* Style text
* switch from selectAll() to select()
* Reflect centroid container's purpose in id
* Remove mass from the output
* Swap to obj
* Add finite check
* Don't draw centroid if no finite values
* Fix finite check
* Remove log
* Toggle label coloring based on colorBy state
* Pass cursor events through centroid svg
* menubar 1
* zoom switching
* centering, pixel perfect canvas
* remove dead args and code
* clipping
* remove log
* if
* connect props
* lint
* undo
* logo left, componetize
* graph back to full height
* shadow to top
* do not prematurely call event handlers during render
* change test to deal with async histogram creation
* left section padding
* lint
* adjust graph to account for top bar,
* lasso tests
* refine histogram tests
* remove testing (onlys)
* enforce column name uniqueness for obs and var
* parameterize the column name containing obs and var user-readable names
* use the new annotation index value from schema
* update f/e unit tests
* PR review suggestions
* lint
* add own range() function
* lodash cleanup
* remove redundant fill range implementations
* remove use of _.get
* sync test babel config with build
* update tests to match new range implementation
* Add numeric inputs for percentiles
* Define initial values for percentile cutoffs in world reducer
* add percentil to crossfilter dimensions
* worldEqUniverse now handles cloned worlds
* add Dataframe.mapColumns
* Wire up handlers for percentile inputs
* World reducer and stateManager know about continuousPercentileMin/Max
* Create world as universe clone (not pointer) to avoid clobbering vals
* Define basic actions for setting continuousPercentileMin/Max
* Under the hood, deal with percentiles between 0 and 1
* Move percentile inputs to visualization settings menu
* Fix padding for undo/redo buttons
* Trigger world rebuild from percentile actions
* BROKEN - pseudocode for clamping dataframe by percentiles upon world rebuild
* fix error handling on clip quantiles; start world clipping implementation
* more unclipped reorg
* rename crossfilter.percentile to quantile
* simplify schema access
* update continuous legend when scale changes
* update color cache when clip changes
* clip obs annotations and var data when clip quantile changes
* use own fromEntries
* fix tests
* stable non-finite float sort/search
* clarify comments
* fix syntax typo
* use new stand-alone clip
* clip expresssion data
* add select tests for non-finite scalars
* basic styles
* clip UI now requires explicit commit
* reset enable/disable accounts for clip percentiles
* better error messages
* fix bug in undo interaction with programatic min brush selection
* small refactoring
* support clipping of int data
* do not perform unnecessary summarizations
* improve caching of dataframe compiled columns
* add percentile precompute to Dataframe.summarize
* use Dataframe.summarize for clip percentiles
* remove obsolete quantile code from corssfilter
* histogram scale and label Y axis, add unclipped X range labels
* layout tweaks
* scatterplot now updates when clip changes
* improve comments
* remove debugging comment
* rework clip number entry validation for usability
* ui tweaks to histogram colors and layout
* enable undo/redo for clip user action
* refine UI on clip value entry
* api cleanup
* update confusing comment
* clarify purpose of isValidDigitKeyEvent
* fix misleading comment
* apply appropriate button-group classes; do not mix span and div
* variable name and comment changes suggested in PR review
* rename sort to sortArray; remove unused and dead code path
* naming changes suggested in PR review
* code review improvements for clarity
* more small changes from PR review
* lint fixes for PR review
* fix spelling error
* clarify that function performs in-place modification of world
* add comment to clarify intent of range operation
* fix bad indents in comments
* clean up __columnsAccessor comments and code
* improve comments around clipPredicate
* field name consistency
* improve comment on quantiles params
Bruce Martin