* disable formatting rules for eslint and add prettier in lint-staged
* update npm modules
* set plugin-proposal-private-methods to loose
* update snapshots due to popover package update
* add missing quotes
Co-authored-by: bkmartinjr <bruce@chanzuckerberg.com>
* __test: create geneset
* example dataset test geneset
* delete geneset test
* edit __test
* gene crud
* Update client/Makefile
Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com>
* copy gene sets separately
* make fix
* ignore test files locally
* csv update
* updated csvs
* fix unit tests for gene set load routes
* add missing fix to czi_hosted unit test
* pin tiledb version, for czi_hosted backend, to <0.9
* Revert tiledb pin to be less than 0.9. Broken tests have been updated in main branch.
* newline, gitignore
* color by and subset
* diffexp sets equal
* add diff exp test class
* fix data class
* diffexp snapshot
* snapshot
* snap3
* snapshot parentInnerhtml
* remove snap
* updated anno snaps
* add test class to gene list div
* new snapshots
* kick off
* Revert "kick off"
This reverts commit 743f551d55.
* remove import
* eol
* revert changes to csv re: gene tests
* global name
Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com>
Co-authored-by: bkmartinjr <bruce@chanzuckerberg.com>
Co-authored-by: Arathi Mani <arathi.mani@chanzuckerberg.com>
Co-authored-by: maniarathi <mani.arathi@gmail.com>
* feat: return two lists for diffexp (#2221)
* sp
* split out derive sort order, tests passing
* sp
* return diff exp results in two lists
* update
* copy implementation over to desktop
* add tests for two lists
* small fixes to complete backend implementation
* accept new diffexp response
* map diff exp response to genesets
* delete )
* name diffexp genesets with population names
* take constants out of state and allow width prop to override
* shorten mini-histo properly truncate and resize depending on expansion
* prepend new genesets
* rename data within diffexp action
* backend
* move diffexp ttest to common code module, update tests
* update for unit tests
* reference actual var
Co-authored-by: Madison Dunitz <madison.dunitz@chanzuckerberg.com>
Co-authored-by: Madison Dunitz <dunitzm@gmail.com>
* update package lock
* allow falsey vals
* remove old diff exp handling
* save diff exp results as geneset
* delete test
* check for undefined or null
* use global geneset description for diffexp
* remove diffexp special code, no longer showing adjpval + logfoldchange
* remove differential map to state
* remove clear from FSM, since we no longer support those actions
* restore controlHelpers test with todo
* first cut at hosted gs routes
* lint
* update tests to match csv parser changes
* update tests to new API
* update gene set name validation rules to match requirements
* add path mapping from dataset to geneset
* add test cases for geneset GET route
* fix test assertion
* remove debugging code
* update gene set uri mapping function
* fix error message
* allow extra user-specified headers in gene set csv file
* clarify comment
* colorby histo
* color graph by mean expression
* move var index after returns
* add genesets as an argument
* varindex
* undo redo for mean expression
* destructure
* ternary
* Revert "destructure"
This reverts commit 2d9432c1c7.
* color by mean for diffexp
* geneset description add
* edit geneset description
* default state for desc
* remove log
* naming, todo
* check for both dup name and desc
* fixes
* do not store gene set modal state in history stack
* Update createGenesetDialogue.js
* Update editGenesetNameDialogue.js
Co-authored-by: bkmartinjr <bruce@chanzuckerberg.com>
* first cut at GET /genesets route
* update existing tests to match code changes
* more GET /genesets and initial tests
* add missing test fixture
* geneset validation accepts OTA format
* genesets route: better error handling, more tests
* lint
* genesets reducer and initial load
* fix lint
* add autosave support for genesets
* remove debug logging
* fix typo
* fix another typo
* update smoke test config for genesets
* smoke test fixes
* more fiddling with smoke tests
* remove single val continous metadata from histogram, add to info drawer
* refactor to save singleContinuous values in state
* fix edge case, single continuous values reappeard in rsb when clipped
This PR adds multiple data to the dataset overview drawer provided by the config endpoint and formats them accordingly. The appearance of this new data is contingent on `dataPortalProps.corpora_schema_version === "1.0.0"`
For QA launch cellxgene with a remixed dataset and click on the button in the upper left-hand corner or the updated button in the info menu.
~~Review opening is blocked by merge of #1805~~
---
Closes#1319
* save tiledb array to s3, dont cache user annotations
* Add option to disable annotation filename prompt (#1787)
Co-authored-by: Madison Dunitz <dunitzm@gmail.com>
* set tiledb default context in cxg_adaptor
Co-authored-by: maniarathi <arathi.mani@chanzuckerberg.com>
Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com>
* Separate userinfo from the config endpoint
previously information about if the user was logged in and their username
was part of the config endpoint.
However, the config endpoint was previously static, and has a cache control.
Rather than not caching the config, a new endpoint called "userinfo"
is created to handle that information.
The config endpoint still has the non-changing part of the authentication:
config:
authentication:
requires_client_login: True/False
login: <uri to login endoint if requires_client_login is True>
logout: <uri to logout endoint if requires_client_login is True>
The userinfo endpoint returns this information:
userinfo:
is_authenticated: True/False
username: <string if is_authenticated>
if authentication is not enabled then the config does not have an authentication key,
and userinfo returns None.
Also in the PR are a few minor code improvements and bug fixes
Co-authored-by: Colin Megill <colinmegill@gmail.com>
* refactor categorical controls state
* lint
* fix race condition in tests
* fix typo
* add missing update on subset
* remove obsolete code
* update jest and puppeteer major version; update all minors
* update when label changes
* remove lint from tests; increase timeouts in e2e tests
* initial refactoring to new async annomatrix
* refine error handling
* fix bad merge
* add continuous legend
* lint
* fix memoization in color table creators
* partial implementation of user defined annotations
* add new annotations action creator file
* first pass at user annotations
* additional user annotation bug fixes
* user annotation auto-save
* unit test cleanup
* lint
* refactor into multiple files
* cleanup
* add column GC
* fix several bugs in user annotations
* remove debug code
* no anonymous functions
* undo redo cleanup
* file cleanup
* scatterplot
* performance
* cleanup
* remove old code
* render in parallel with load
* fix race condition
* simply graph rendering
* render throttle DRY
* fix category label order
* fix typo in e2e test setup
* re-fix the e2e test setup
* be more tolerant of races
* anno matrix unit tests
* temp disable reembedding
* pilot port continuous histo to react-async
* name change
* lint
* fix repaint bug
* typo fix
* update snap to match new ids
* world/universe name cleanup
* move annoMatrix to src dir
* use private underscore naming convention
* fix corner case in all selected
* name cleanup
* add layout control
* init edge case
* lint
* port scatterplot
* fix label indexing bug and improve tests
* port category to react-async
* fix user annotation labelling while subset
* select all of prev layout on layout switch
* fix race with crossfilter update
* prettier lint
* fix misleading comment
* fix url composition in loader
* first pass at crossfilter tests
* lint
* lint
* fix typo
* improved error handling for network errors
* fix memoization bug
* add memo
* refactor for performnce
* add missing single-value handling in select exact parser
* small bugs discovered by tests
* lint
* additional crossfilter unit tests
* remove extraneous comment
* add support for automatic category determination
* lint
* fix render bug in category
* take advantage of schema categories guarantee
* lint
* do not clear history when resetting
* enhanced annomatrix gc
* lint
* finish renaming to follow conventions; fix clone race bug
* lint
* add priority based loading to improve initial data load UX
* crossfilter cache perf
* perf tuning
* remove timers
* documentation
* PR review changes
* PR review changes
* more PR review edits
* improve clarity of comment
* more PR review fixes
* port centroidLabels to use react-async
* remove dead code
* pr review updates
* oops, remove logging
* add case to removing labels
* add disabled state to centroid button
* fix centroid test button clicking order
* don't accidentally un-toggle first color
* run eslint --fix
* camelcase
* camelCase config part 1
* part 2
* part 3 - removing subscripts
* fix "class-methods-use-this"
* fix "class-methods-use-this"
* fix eslint ignores
* add eslint ignore for set state in update
* reformat comments to appease eslint
* add a11y features
* sort-comp fix
* a11y fix
* add ignore for set state in update
* add a11y htmlFor
* remove unused toast
* remove unnecessary bind
* add ignore for set state in update
* add rel="noopener noreferrer"
Using target="_blank" without rel="noopener noreferrer" is a security risk: see https://mathiasbynens.github.io/rel-noopener
* use arrow function to bind
* remove unused definitions/declarations
* prettier
* remove unused state
* add comments to empty catch blocks remove curly brackets
* escape '
* use eqeqeq
* switch from default export
* remove ignore log
* remove static
* fix import
* revert subscripting config
* clean-up
* remove unnecessary subscript
* fix new errors from master
* change category click handler to a class property
* fix camelcase changes that slipped by
* unused import
* Fix newly introduced ESLint errors from addGenes
* Refactor CSS layout and react logic for layout
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1022
* Menubar should wrap inside middle pane instead of overlapping left
sidebar when window is scrunched
* cellxgene should have a minimum width of 1240px
1. Replace absolute positioning and dimension calculation with css grid
2. Use flexbox for wrapping menubar buttons
* Middle pane (graph) can calculate its own size
* Removing components calculating their size/position relative to
eachother increases modularity, decreases use of global variables
* Improved some scrollbar behavior
* Removed responsive reducer, propagating window size to components
triggers unnecessary events and encourages breaking modularity; doing
this made some components state agnostic
Reference: https://css-tricks.com/snippets/css/complete-guide-grid/
* Reposition the continuous legend
* Small fixes
* Respond to feedback from @colinmegill
* Respond to feedback from @colinmegill
Add more documentation on the renderGraph method.
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1433
When selection is deselected the world is reset to the universe the
crossfilter is reset to that in the resetCache, including the embeddings
in the layout_XY dim. However, the embedding selection stays the same.
If the embedding selected is not the default, the embedding shown to the
user will be different than the embedding layout_XY in the crossfilter,
causing lasso selections to be made against the wrong embedding
coordinates
* Add user-defined category-label colors
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1152
As described in https://github.com/chanzuckerberg/cellxgene/issues/1307
* Respond to feedback from @bkmartinjr in nodejs
* Respond to feedback from @bkmartinjr in python
* Add tests to the server module
* Autoformat python, run linter
* Make colors_get error handling specific
* Respond to feedback from @bkmartinjr
* Respond to feedback from @bkmartinjr
* Fix whitespace
* Fix python lint errrors
* Update documentation
* Add --disable-user-colors option to launch and cxgtool.py
* Fix python formatting
* Rename '--disable-user-colors' to '--disable-custom-colors'
* first cut at re-embedding route and back-end support
* update and expand config route tests
* add scanpy_umap
* add reembedding to config route parameters
* front-end support for reembedding fetch and UI
* remove unused imports
* add loading state
* save reembedding in reducer state
* improve withColsFrom
* transmit reembed schema to client; pick unique embedding names
* display embeddings
* format
* lint
* spaces, tab size 2
* lint
* test hack for smoke-test race
* back out hack sleep
* add check for backed mode
* add unit test for reembedding
* lint
* hide re-embedding CLI param from help
* Remove unused variables and imports
* Simplify conditional
* Fix typo
* Do not overprune var data cache
There is a bug in how the universe and world gene sets are constructed
and passed to `ControlsHelpers.pruneVarDataCache` that causes the var
data cache to be over-pruned. This commit fixes the issue.
Consider the following example from the node console:
```
❯ node
Welcome to Node.js v13.5.0.
Type ".help" for more information.
> new Set([1], [2], [3])
Set(1) { 1 }
```
What we really want is the set `Set(3) { 1, 2, 3 }`, which can be
constructed as:
```
> new Set([].concat([1], [2], [3]))
Set(3) { 1, 2, 3 }
```
* Disable ColorBy button for truncated categories
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1156
For categories that have more than 100 labels we truncate the labels in
the UI, but still allowed users to ColorBy these categories. Coloring by
these categories can cause browsers to get bogged down.
This commit disables ColorBy for truncated categories.
* Minor documentation spelling and typo fixes
* Respond to feedback from @liaprins-czi
* Respond to feedback from @colinmegill and @bkmartinjr
* Undo selection appends diffExp genes to user gene list
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1171
Need: When a user performs a differential expression from within
a sub-selection (world) of the data and then resets the selection to all
cells (universe), the differential expression results are no longer
valid.
Approach:
* When the selection is reset, move the top (maxUserDefinedGenes
- len(userDefinedGenes) from the differential expression results to the
list of user defined genes
* Raise maxUserDefinedGenes to 25 to give users more room and
accommodate the extra genes transferred in from differential expression
Other commits:
* Choose different button icons
* Add diff exp genes to user defined genes on subset too
* Respond to feedback from @liaprins-czi and @bkmartinjr
* add simple error message helper
* port all label name pickers to use the new LabelInput component
* use pure components where possible
* cleanup
* more cleanup
* lint
* change new label prompt
* Add undo/redo tests for annotations
Fixes https://github.com/chanzuckerberg/cellxgene/issues/969
... also refactor the tests for DRY.
* Add done()
* Make e2e annotations tests safer to concurrency
* Add data-testclass for save state.
* Simplify tests and make them dependent on save state
* load annotations individually
* fix type check to be more general
* update node CI version from 10 to 12
* node 11
* debug print node version
* travis node version to latest
* try nvm
* remove extraneous node_js statement
* remove node version debugging printf
* incrementally load all annotations and layout
* process annotations and layout as they are loaded
* fix tests
* sort categories incrementally
* incrementally build category view summary; add category loading spinner
* add spinner to continuous metadata
* configure undoable reducer
* incremental crossfilter creation
* improve busy layout
* more layout cleanup
* correctly reconcile categories in schema
* refine layout of lsb spinners
* more spinner layout work
* more spinner layout
* always load layout before obs annotations
* Undo feature: fix case where previous state has no state filter
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1099
When the previous state that the undo feature is trying to roll back to
has no filter state, merging javascript dictionaries result in keeping
the current state filter, preventing the actionFilter from saving the
new state.
* Fix whitespace
* add sample ontologies file
* add ontologies reducer
* Move select category to own component
* Dialog and Input factored out
* refactoring categorical, partway
* validationn
* anno
* suggest populates input
* frontend for ontology working
* initial implementation of back-end support for ontologies
* edit is now dialog again
* autosuggest working on edit
* part way through create arbitrary label
* handle choice in function
* pass duplicate cat prop
* editing works
* update test to match new CLI params
* fix occupancy alignment
* edit category as dialogue
* secondary button
* remove stubbed out ontologies
* add label setting upon new label creation
* Update legal characters for labels (#1119)
* Allow any term in the ontology (bypass legal name check)
* Add hyphens and parens to legal characters in names
* improve performance for large ontologies
* correctly handle case where ontologies are disabled
* fix logic error in CLI
Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>
* PR cleanup 1
* lint
* validate user generated labels
* finish hooking up connected suggest component
* protect against undefined callbacks
* Fix illegal characters error message
* break out npm run commands
* fix error detection on label edit
Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>
Co-authored-by: Sidney Bell <sidneymbell@users.noreply.github.com>
* refactor reducer to no longer support hover state and hold many labels
* refactor to generate centroidCoordinates for all values of a category
* create hash for function and memoize export
* create button to display all labels for a category
* clear state
* create label for each thing
* calculate on each value
* change to in place modification of map
* switch to for loop with iterator instead of forEach
* use map from centroidLabel instead of creating copy
* adapt for map
* utilize tarrays
* begin documentation
* disable centroids if in zoom mode
* clean up
* persist uncalc coordinates
* document
* clean up and document
* cleanup and document
* fix
* fix undefined labels and document changes
* fix first element skip
* fix conditional recalc
* rename centroidLabel -> centroidLabels
* break out dilation on hover to new reducer
* numerous styling changes for readability
* change centroid icon
* remove colorAccessor from parameters
* recalc centroids on world change
* make label toggle undoable
* remove unused import
* highlight labels on hover
* remove special characters from svg id
* lighten backdrop
* only generate new centroids if they pre-exist
* fix issue with spaces in catagorical value name
* add label buttons to menubar
* change reducer to use colorAccessor and have single toggle
* fix check to see if svg should be rendered
* move svg overlays onto a single svg layer
* dilate on label hover
* remove logs
* allow centroid to update along side regl renders
* allow actions to pass through svg if in zoom mode
* remove artifact from circle
* remove comment
* remove disabling of centroid button
* fix conditional map to screen
* make styling label conditions stricter
* prettier
* refactor onto master
* refactor computePointFlags() to use pointDilation store
* notify when viewport changes
* move svg attributes out of lasso setup and prevent rerenders/writes
* begin playing with transform matrix
* first solution for camera interaction
* create transform using nested groups
* semi-working method using nested groups with transforms
* inversely scale text
* properly do final transform
* cleanup dead / test code
* reinstate original functionality
* breakout centroid labels labels into separate component
* default toggle on for testing
* separate lasso and centroid layers
* remove unnecessary attributes, working hover
* dilation on label hover
* fix dilation on scatterplot
* add dilation on label hover
* break overlay into separate component
* make overlay agnostic to children
* move label mouse actions to centroidlabels component, add overlay state
* remove lasso on switch to camera
* disallow user selection
* fix reducer
* fix subset with continuous color error
* reset labels on color by continuous
* revert centroids on by default
* refactor for nested restructuring
* remove update checking
* remove unused method
* readd deleted hover delay
* remove old centroid setup
* remove centroid from undoable
* cleanup dead code
* remove dead code
* rollback unnecessary changes
* begin adding annotation functionality
* add annotation functionality
* add reset and undo functionality
* change centroids on layout change
* don't create label for unassigned
* add comment pointing out POI for performance
* touch up matrix transform comment
* add comment explaining coordinate space and children's assumed space
* remove dead code
* switch to pure component
* connect centroidLabels to redux
* clean up camera check and null result
* tool tip change
* rename centroid toggle and the like
* fix the misalignment of buttons, also make blueprint use consistent
* fix comment spelling mistakes
* introduce variable for cleaner logic expressions and state assignment
* alter tooltip text to back color by interaction
* remove manual iterator manipulation for forEach()
* remove debounce
* nit fix
* tooltip wording fix
* lint
