Commit Graph
95 Commits
Author SHA1 Message Date
Severiano Badajoz 4e96847032 Create Truncation Component (#1500)
* remove ESLint rules

operator-linebreak is no different then default
quotes is overwriting prettier config

* add string-pixel-width dep

* don't lint-staged src, only staged

* add widthMap

* create Truncate component

* refactor in truncate component

* add font load checking

* remove font-family styling

* render Truncate's child instead of creating own component to render

* refactor to use Truncate component

* add span back

* support children

* remove maybeTruncateString

* sub in Truncate component

* add bold prop

* accurately compute largest possible string

* remove logs

* tweak truncation method

* memoize comp function

* explain disable

* fix bugs w/ abs/floor

* tweak widths

* tweak widths

* fix font size

* remove border

* move test-id

* attempt css solution

* Revert "attempt css solution"

This reverts commit aac4d8a6f6.

* CSS solution v2

* remove string-pixel-width

* remove widthsMap

* remove dead code

* remove "data-truncated" as it is always true

* tweak label width

* fix e2e tests

* remove testing string

* e2e annotations tweaks

* correct snapshot

* remove resolves

* check for labels

* add test-id

* format fix

* update snapshot

* fix color

* pull constants out where available
2020-06-05 10:14:42 -07:00
Bruce Martin 2ba4944f5c initial bug fixes and test improvements for the matrix refactor (#1503)
* initial bug fixes and test improvements for the matrix refactor

* lint
2020-06-02 09:47:40 -07:00
Severiano Badajoz 271e7a1e76 Fix prettier related ESLint issues (#1493)
* add quotes

* add prettier dev-dep

* run prettier on src

* add eslint-prettier-plugin to install
2020-05-26 09:33:59 -07:00
Matt Weiden 8d762e59ca Deduplicate merged gene list in subsetAndResetGeneLists (#1490)
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1486
2020-05-20 15:35:27 -07:00
Severiano Badajoz c34a68304e remove all linting errors on client/src (#1463)
* run eslint --fix

* camelcase

* camelCase config part 1

* part 2

* part 3 - removing subscripts

* fix "class-methods-use-this"

* fix "class-methods-use-this"

* fix eslint ignores

* add eslint ignore for set state in update

* reformat comments to appease eslint

* add a11y features

* sort-comp fix

* a11y fix

* add ignore for set state in update

* add a11y htmlFor

* remove unused toast

* remove unnecessary bind

* add ignore for set state in update

* add rel="noopener noreferrer"

Using target="_blank" without rel="noopener noreferrer" is a security risk: see https://mathiasbynens.github.io/rel-noopener

* use arrow function to bind

* remove unused definitions/declarations

* prettier

* remove unused state

* add comments to empty catch blocks remove curly brackets

* escape '

* use eqeqeq

* switch from default export

* remove ignore log

* remove static

* fix import

* revert subscripting config

* clean-up

* remove unnecessary subscript

* fix new errors from master

* change category click handler to a class property

* fix camelcase changes that slipped by

* unused import

* Fix newly introduced ESLint errors from addGenes
2020-05-19 12:38:29 -07:00
Colin Megill 9026e0ce41 Sci notation on continuous legend (#1450)
* sci notation on continuous legend

* significant digits
2020-05-07 12:59:10 -04:00
Severiano Badajoz 6cccc41c0f run prettier(2.0.5) (#1438) 2020-05-04 10:26:42 -07:00
Matt Weiden 546e272a60 Add user-defined category-label colors (#1402)
* Add user-defined category-label colors

Fixes https://github.com/chanzuckerberg/cellxgene/issues/1152

As described in https://github.com/chanzuckerberg/cellxgene/issues/1307

* Respond to feedback from @bkmartinjr in nodejs

* Respond to feedback from @bkmartinjr in python

* Add tests to the server module

* Autoformat python, run linter

* Make colors_get error handling specific

* Respond to feedback from @bkmartinjr

* Respond to feedback from @bkmartinjr

* Fix whitespace

* Fix python lint errrors

* Update documentation

* Add --disable-user-colors option to launch and cxgtool.py

* Fix python formatting

* Rename '--disable-user-colors' to '--disable-custom-colors'
2020-04-26 22:52:57 -07:00
Bruce Martin a3df1a9f4a performance and correctness fixes to brushable histogram (#1390) 2020-04-13 20:03:46 -07:00
Bruce Martin 136093d583 add GET routes for expression data (#1387)
* add GET routes for expression data

* fix comment typo
2020-04-13 11:28:35 -07:00
Severiano Badajoz 4efabf5231 refactor to remove non-standard terms in centroid files (#1330)
* refactor to remove non-standard terms

* switch key/value -> label/coords

* fix missing parenth
2020-04-06 10:20:11 -07:00
Bruce Martin de8033e551 correctly handle categoricals that are not strings (#1346) 2020-04-03 15:50:02 -07:00
Bruce Martin 7c56041b11 categorical type handling fix (#1342)
* fix numeric category conversion bug

* correctly compute categorical summaries

* lint

* remove debugging print
2020-04-02 14:44:07 -07:00
Bruce Martin 708a5af039 Add fetch concurrency limit on obs annotation loading (#1318)
* add fetch concurrency limit

* add tests for PromiseLimit
2020-03-30 16:23:09 -07:00
Severiano Badajoz b249df9036 add/fix label tooltips (#1278)
* Move tooltip to right

* Change function to only return truncated string

* add tooltip to category label

* use position instead of PopoutPosition
2020-03-23 10:28:54 -07:00
Severiano Badajoz 3ba4cf4483 fix bug created from master rebase (#1216) 2020-03-11 15:20:20 -07:00
Severiano Badajoz 3bf72829c2 pass empty coordinates when a continuous annotation is colored (#1212) 2020-03-11 13:01:14 -07:00
Severiano Badajoz 7cd9a0032a optimize centroid util (#1147)
* move unvarying evaluations outside of loop

* refactoring

* remove perf checks

* commenting

* minor fix + renaming

* small fix

* benchmarking

* merge master

* Revert pref checks

* renaming and comment

* remove redundant sets in Map
2020-03-11 12:35:42 -07:00
Colin MegillandMatt Weiden 234f25b782 Conditionally truncate category string (#1206)
* maybe truncate string

* add string formatting to test

* correct import

* destructuring

* add maxlength

* test

* Respond to feedback from @bkmartinjr

Co-authored-by: Matt Weiden <538456+mweiden@users.noreply.github.com>
2020-03-10 18:14:26 -07:00
Bruce Martin 144b19c449 experimental re-embedding (#1186)
* first cut at re-embedding route and back-end support

* update and expand config route tests

* add scanpy_umap

* add reembedding to config route parameters

* front-end support for reembedding fetch and UI

* remove unused imports

* add loading state

* save reembedding in reducer state

* improve withColsFrom

* transmit reembed schema to client; pick unique embedding names

* display embeddings

* format

* lint

* spaces, tab size 2

* lint

* test hack for smoke-test race

* back out hack sleep

* add check for backed mode

* add unit test for reembedding

* lint

* hide re-embedding CLI param from help
2020-03-09 16:53:30 -07:00
Matt Weiden bdd69160e2 Do not over-prune var data cache (#1198)
* Remove unused variables and imports

* Simplify conditional

* Fix typo

* Do not overprune var data cache

There is a bug in how the universe and world gene sets are constructed
and passed to `ControlsHelpers.pruneVarDataCache` that causes the var
data cache to be over-pruned. This commit fixes the issue.

Consider the following example from the node console:

```
❯ node
Welcome to Node.js v13.5.0.
Type ".help" for more information.
> new Set([1], [2], [3])
Set(1) { 1 }
```

What we really want is the set `Set(3) { 1, 2, 3 }`, which can be
constructed as:

```
> new Set([].concat([1], [2], [3]))
Set(3) { 1, 2, 3 }
```
2020-03-05 18:04:14 -08:00
Matt Weiden 1547a864c2 Disable ColorBy button for truncated categories (#1191)
* Disable ColorBy button for truncated categories

Fixes https://github.com/chanzuckerberg/cellxgene/issues/1156

For categories that have more than 100 labels we truncate the labels in
the UI, but still allowed users to ColorBy these categories. Coloring by
these categories can cause browsers to get bogged down.

This commit disables ColorBy for truncated categories.

* Minor documentation spelling and typo fixes

* Respond to feedback from @liaprins-czi

* Respond to feedback from @colinmegill and @bkmartinjr
2020-03-04 09:19:12 -08:00
Matt Weiden 05323ae643 Undo selection appends genes from differential expression to user gene list (#1183)
* Undo selection appends diffExp genes to user gene list

Fixes https://github.com/chanzuckerberg/cellxgene/issues/1171

Need: When a user performs a differential expression from within
a sub-selection (world) of the data and then resets the selection to all
cells (universe), the differential expression results are no longer
valid.

Approach:
* When the selection is reset, move the top (maxUserDefinedGenes
- len(userDefinedGenes) from the differential expression results to the
list of user defined genes
* Raise maxUserDefinedGenes to 25 to give users more room and
accommodate the extra genes transferred in from differential expression

Other commits:
* Choose different button icons
* Add diff exp genes to user defined genes on subset too
* Respond to feedback from @liaprins-czi and @bkmartinjr
2020-03-02 12:07:12 -08:00
Bruce Martin bf7d7342d5 Improve label picking (#1179)
* add simple error message helper

* port all label name pickers to use the new LabelInput component

* use pure components where possible

* cleanup

* more cleanup

* lint

* change new label prompt
2020-02-28 15:46:40 -07:00
Severiano Badajoz 0043308b52 add centroid util unit tests (#1172)
* scaffold the test

* create tests
2020-02-26 12:16:13 -08:00
Bruce Martin fb1f0c6469 fix latent crossfilter bug (#1162) 2020-02-21 14:14:41 -07:00
Matt Weiden d30087ca63 Add frontend smoke tests for annotations (#1154)
* Add smoke test for annotations features

* Do not save during annotations tests

* Fix botched rebase in dev guidelines

* Revert "Do not save during annotations tests"

This reverts commit f0bd970bb2.

* Respond to feedback from @bkmartinjr
2020-02-14 17:23:45 -08:00
Bruce Martin e770db1e2c load annotations incrementally (#1107)
* load annotations individually

* fix type check to be more general

* update node CI version from 10 to 12

* node 11

* debug print node version

* travis node version to latest

* try nvm

* remove extraneous node_js statement

* remove node version debugging printf

* incrementally load all annotations and layout

* process annotations and layout as they are loaded

* fix tests

* sort categories incrementally

* incrementally build category view summary; add category loading spinner

* add spinner to continuous metadata

* configure undoable reducer

* incremental crossfilter creation

* improve busy layout

* more layout cleanup

* correctly reconcile categories in schema

* refine layout of lsb spinners

* more spinner layout work

* more spinner layout

* always load layout before obs annotations
2020-02-10 11:22:27 -08:00
d48647a655 Ontologies (#1110)
* add sample ontologies file

* add ontologies reducer

* Move select category to own component

* Dialog and Input factored out

* refactoring categorical, partway

* validationn

* anno

* suggest  populates input

* frontend for ontology working

* initial implementation of back-end support for ontologies

* edit is now dialog again

* autosuggest working on edit

* part way through create arbitrary label

* handle choice in function

* pass duplicate cat  prop

* editing works

* update test to match new CLI params

* fix occupancy alignment

* edit category as dialogue

* secondary button

* remove stubbed out ontologies

* add label setting upon new label creation

* Update legal characters for labels (#1119)

* Allow any term in the ontology (bypass legal name check)

* Add hyphens and parens to legal characters in names

* improve performance for large ontologies

* correctly handle case where ontologies are disabled

* fix logic error in CLI

Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>

* PR cleanup 1

* lint

* validate user generated labels

* finish hooking up connected suggest component

* protect against undefined callbacks

* Fix illegal characters error message

* break out npm run commands

* fix error detection on label edit

Co-authored-by: Bruce Martin <bruce@chanzuckerberg.com>
Co-authored-by: Sidney Bell <sidneymbell@users.noreply.github.com>
2020-01-23 17:04:17 -05:00
Severiano Badajoz 8d725b1ad9 centroid labels (#872)
* refactor reducer to no longer support hover state and hold many labels

* refactor to generate centroidCoordinates for all values of a category

* create hash for function and memoize export

* create button to display all labels for a category

* clear state

* create label for each thing

* calculate on each value

* change to in place modification of map

* switch to for loop with iterator instead of forEach

* use map from centroidLabel instead of creating copy

* adapt for map

* utilize tarrays

* begin documentation

* disable centroids if in zoom mode

* clean up

* persist uncalc coordinates

* document

* clean up and document

* cleanup and document

* fix

* fix undefined labels and document changes

* fix first element skip

* fix conditional recalc

* rename centroidLabel -> centroidLabels

* break out dilation on hover to new reducer

* numerous styling changes for readability

* change centroid icon

* remove colorAccessor from parameters

* recalc centroids on world change

* make label toggle undoable

* remove unused import

* highlight labels on hover

* remove special characters from svg id

* lighten backdrop

* only generate new centroids if they pre-exist

* fix issue with spaces in catagorical value name

* add label buttons to menubar

* change reducer to use colorAccessor and have single toggle

* fix check to see if svg should be rendered

* move svg overlays onto a single svg layer

* dilate on label hover

* remove logs

* allow centroid to update along side regl renders

* allow actions to pass through svg if in zoom mode

* remove artifact from circle

* remove comment

* remove disabling of centroid button

* fix conditional map to screen

* make styling label conditions stricter

* prettier

* refactor onto master

* refactor computePointFlags() to use pointDilation store

* notify when viewport changes

* move svg attributes out of lasso setup and prevent rerenders/writes

* begin playing with transform matrix

* first solution for camera interaction

* create transform using nested groups

* semi-working method using nested groups with transforms

* inversely scale text

* properly do final transform

* cleanup dead / test code

* reinstate original functionality

* breakout centroid labels labels into separate component

* default toggle on for testing

* separate lasso and centroid layers

* remove unnecessary attributes, working hover

* dilation on label hover

* fix dilation on scatterplot

* add dilation on label hover

* break overlay into separate component

* make overlay agnostic to children

* move label mouse actions to centroidlabels component, add overlay state

* remove lasso on switch to camera

* disallow user selection

* fix reducer

* fix subset with continuous color error

* reset labels on color by continuous

* revert centroids on by default

* refactor for nested restructuring

* remove update checking

* remove unused method

* readd deleted hover delay

* remove old centroid setup

* remove centroid from undoable

* cleanup dead code

* remove dead code

* rollback unnecessary changes

* begin adding annotation functionality

* add annotation functionality

* add reset and undo functionality

* change centroids on layout change

* don't create label for unassigned

* add comment pointing out POI for performance

* touch up matrix transform comment

* add comment explaining coordinate space and children's assumed space

* remove dead code

* switch to pure component

* connect centroidLabels to redux

* clean up camera check and null result

* tool tip change

* rename centroid toggle and the like

* fix the misalignment of buttons, also make blueprint use consistent

* fix comment spelling mistakes

* introduce variable for cleaner logic expressions and state assignment

* alter tooltip text to back color by interaction

* remove manual iterator manipulation for forEach()

* remove debounce

* nit fix

* tooltip wording fix

* lint
2020-01-22 13:40:22 -08:00
Matt Weiden ddbfc458d7 Remove d3 dependency in crossfilter (#1108)
* Remove d3 dependency in crossfilter

Fixes https://github.com/chanzuckerberg/cellxgene/issues/648

* Revert "Remove d3 dependency in crossfilter"

This reverts commit 03b478f7dc.

* Remove d3 dependency in crossfilter

Fixes https://github.com/chanzuckerberg/cellxgene/issues/648

* Fix typo in docstring
2020-01-14 13:46:53 -08:00
Bruce Martin 677433bbf5 Fix label sorting bugs (#1102)
* move category label sort to utils

* refactor cat label sort

* fix category label sort and color assignment

* convert whitespace from tabs to spaces
2020-01-09 13:34:25 -08:00
Matt Weiden 4c1ecf9e52 Specify histogram thresholds in fixed-length array (#1086)
* Specify histogram thresholds in fixed-length array

Fixes https://github.com/chanzuckerberg/cellxgene/issues/1082

For background, see the following:
* https://github.com/d3/d3-array/issues/46
* https://stackoverflow.com/questions/15880058/d3-js-ticks-function-giving-more-elements-than-needed

* Oops, off by one!
2020-01-06 11:13:02 -08:00
Bruce Martin a593e95ab3 annotations CLI and file UX rework (#1049)
* rename config param label-file

* annotations rework - CLI params, file naming and backups

* lint

* improve cli option error checks

* enable session cookies

* enable session cookies

* add session id

* name annotations file in multi-dataset and multi-user safe manner

* pass data user hash to front-end

* add annotation collection name support to front-end

* add constant for annotation data collection name

* parameterize annotation collection name; make it sticky in the session

* clarify comments

* hard wire a temporary data collection name for testing

* prettier

* test comment

* package command

* set annotations  filename dialog

* name  and hash are visible

* wire up data collection capture
2019-11-25 15:28:28 -08:00
Bruce Martin 5713aa8792 category and label name validation updates (#1056)
* update label validation with new schema

* add new error messages for new label errors; general cleanup
2019-11-22 10:32:00 -08:00
Bruce Martin ee62dd355f [WIP] JS lint and dead code removal (#1053)
* lint and dead code removal

* fix regressions
2019-11-21 14:10:02 -08:00
Bruce Martin 9a540a4f0a fix erroneous state clearing (#1052) 2019-11-19 11:11:19 -08:00
Bruce Martin a52e86a69b improve category and label name validation (#1034) 2019-11-13 09:40:37 -08:00
Bruce Martin 7599af451f add context sensitivity to annotation "label" menu item (#1015)
* cache common results to reduce jank

* add util function in support of annotation labelling

* enable/disable annotation edit menus per issue 972; improve formatting

* PR review requests
2019-11-04 14:22:38 -08:00
Bruce Martin a16c2e2747 crossfilter immutability error (#1009)
* add rename immutability test

* make rename immutable
2019-11-04 14:14:17 -08:00
Bruce Martin 3f2811d9da rendering performance improvements (#968)
* freeze objects

* component rendering perf work

* use PureComponent where safe

* remove obsolete WorldUtil code

* make brushable histogram a pure component
2019-10-04 07:59:22 -07:00
Bruce Martin 3660a6cc27 Experimental - manual annotations (#837)
* icons, partway

* redux for values

* onChange

* cancel

* annotations lifecycle for category names

* copy categorical

* edit category

* add Dataframe.withColsFrom

* render user annotations; default add/delete annotation category

* add label name to actions

* category name edit

* error checking improvements

* change schema field isUserAnnotation to writable

* always have an unassigned label; implement delete label

* implement add new label and edit label name

* label current cell selection

* fix select exact bug in crossfilter

* clean up categorical reducer

* fix tests

* remove debugging printf

* implement subset/reset for user annotations

* undo redo support for user annotations

* remove duplicate button from categories

* add modal

* remove obsolete duplicate annotation reducers

* remove old debugging printf

* connect modal to annotation create and dup

* initial full-stack wiring

* finish up end-to-end wiring

* fix existing unit tests

* fix pytests to match new schema API

* remove debugging printfs

* add label file rotation

* remove obsolete comment

* add fbs encode/decode tests

* add tests for writable annotations

* simplify code

* fix hashing bug with FBS encoding

* lint

* fix smoke tests

* improve error checking in Dataframe.withColsFrom

* add unit test for Dataframe.withColsFrom

* add unit test for Dataframe.columns and Dataframe.renameCol

* fix bug in FBS encode, add better error checks, refactor

* add FBS encode/decode test

* add clarifying comment

* clean up action type names; fix state inconsistency in crossfilter update

* change autosave timer to 2.5sec

* sort categorical metadata render order so it remains consistent

* add temporary autogenerated label for add-new-label operation

* fix hover-over label menu interference with cell highlighting

* remove debugging code

* add missing reducer cases & fix typo

* make dataframe memoize more general purpose

* add dev mode for annos

* fix error on select duplicate

* handle zero occupancy categories

* correctly maintain unclipped AND clipped world

* correctly handle zero length FBS matrix and label files

* ensure all writable categorical schema contains an unassigned category

* handle case where building occupancy stack for category with no members

* dialog for creating label, disable button if duplicate or empty

* visually separate writeable

* edit category

* fix edit category name

* remove debugging code

* fix edit annotation label

* visually define unassigned, change options

* Pull in requirements.txt from `master`

* label currently selected cells

* duplicate label

* lint

* fix pytest merge issues

* rename --label-file to --experimental-label-file

* remove debugging console log

* spelling error fix; fix bug found in PR review.

* lint
2019-09-18 07:33:41 -04:00
Bruce Martin c35f94fe6d add constraints on pan and zoom-over point (#904) 2019-08-28 17:56:46 -07:00
Bruce Martin b0562a11fd scale point size for graph and scatter plots (#903)
* scale point size for graph and scatter plots

* improve comments
2019-08-27 14:12:53 -07:00
Bruce Martin 0f36b49840 Graph/scatterplot improvements (#893)
* many graph/scatterplot fixes: 722, 882, 758, 886

* fix e2e test breakage caused by graph transform work

* convert webgl point shaders to use flags rather than explicit point size

* remove unused packages from dependencies

* fix typo in regl prop name

* factor common code into util helper file

* legibility tweaks
2019-08-22 20:56:05 -07:00
Bruce Martin 906d65c06f fix binning error in continuous data histogram (#869) 2019-08-05 10:40:56 -07:00
Severiano Badajoz ca20add577 reset colorAccessor and colorMode if colored diffexp gene is removed (#843)
* add action to clear colorMode and colorAccessor if diffexp is removed

* create new colorHelper function

* creater colorHelper for conditionally setting state

* add abbr

* revert abbr
2019-07-11 14:12:38 -07:00
Severiano Badajoz 941c297363 categorical vs continuous mini histograms (#827)
* comment

* add histogram functionality to Dataframe; port category occupancy to use it

* fix binning and create histogram for continous by catagorical

* Remove unnecessary logs

* Begin work on KDE

* Replace broken KDE with working histogram

* Define domain and range based on data from histogram

* Fix occupancy

* Add continuous obs and switch to canvas

* Stop value from always rerendering

* clear before render

* Clear canvas on render

* refactor categorical occupancy to canvas

* Remove log

* simplify finding max

* refactor kde->histogram and occupancy->bins

* refactor svg -> canvas

* rename to occupancy stack

* create popup

* add metadata and categorical values to popup

* fix overflow

* remove zeros info

* style graph

* fix shouldComponentUpdate to look for world changes

* change categorySelected -> categoryValueSelected

* refactor out render

* remove comment

* conditionally have bottom border

* remove diff comp

* remove comments

* remove unnecessary mapping

* Add comments describing drawing functions

* comments

* flip comparison order

* remove logging

* move default to parameter

* move defaults to parameter

* disable popover if not showing histogram

* fix wording and styling

* add line break
2019-07-09 11:19:01 -07:00
Severiano Badajoz 334b8bb8da draw labels marking the centroids of category value clusters (#809)
* Connect mouse over events to reducer actions

* Change Styling on hover

* Rename reducer actions to be more descriptive

* Reorder reducer in cascade

* Create centroid calculation util

* Whitespace

* Typo fix, use correct action

* Create centroid calc util

* Create centroid svg setup

* Refactor existing svg layer to toolSVG

* Change calcCentroid signature and centroidXY to match mapPointToScreen

* Add id and styling

* Run prettier

* Set z-index to 999

* Draw the label

* Introduce the centroid SVG, refactor code to allow both SVG layers

* Add text label and compute radius based on population

* Implement optional chaining

* Update font family

* Optimize calcMeanCentroid()

* Create and utilize calcMedianCentroid()

* Remove mass circle from label

* Remove styling change on hover

* Remove reducer action logs

* Prettier

* Swap out binds for arrow functions

* Style text

* switch from selectAll() to select()

* Reflect centroid container's purpose in id

* Remove mass from the output

* Swap to obj

* Add finite check

* Don't draw centroid if no finite values

* Fix finite check

* Remove log

* Toggle label coloring based on colorBy state

* Pass cursor events through centroid svg
2019-06-13 15:40:57 -07:00
Bruce Martin 3dc45d6330 do not hard-wire column names in annotations (#785)
* enforce column name uniqueness for obs and var

* parameterize the column name containing obs and var user-readable names

* use the new annotation index value from schema

* update f/e unit tests

* PR review suggestions

* lint
2019-05-24 21:00:54 -07:00