Commit Graph

81 Commits

Author SHA1 Message Date
Colin Megill
03bad04436 Embedding button to lower left, cell selection (#1658)
* embedding

* menu bottom left

* button

* change gutters to support lower toolbar

* fix scatterplot layout

* fix tests to match new layout

* fix smoke tests to match new layout

* better sentence, dataset.nObs to top

* scatterplot position

Co-authored-by: bkmartinjr <bruce@chanzuckerberg.com>
2020-07-22 18:48:21 -04:00
Bruce Martin
2715157793 categorical UI improvements (#1649)
* return undefined for unknown fields

* memoize fetch response

* handle rename of user annotation

* performance and cleanup

* fix snapshots

* fix comment

* lint
2020-07-20 08:52:41 -07:00
Bruce Martin
1269e188be Redux refactor (#1571)
* refactor categorical controls state

* lint

* fix race condition in tests

* fix typo

* add missing update on subset

* remove obsolete code

* update jest and puppeteer major version; update all minors

* update when label changes

* remove lint from tests; increase timeouts in e2e tests

* initial refactoring to new async annomatrix

* refine error handling

* fix bad merge

* add continuous legend

* lint

* fix memoization in color table creators

* partial implementation of user defined annotations

* add new annotations action creator file

* first pass at user annotations

* additional user annotation bug fixes

* user annotation auto-save

* unit test cleanup

* lint

* refactor into multiple files

* cleanup

* add column GC

* fix several bugs in user annotations

* remove debug code

* no anonymous functions

* undo redo cleanup

* file cleanup

* scatterplot

* performance

* cleanup

* remove old code

* render in parallel with load

* fix race condition

* simply graph rendering

* render throttle DRY

* fix category label order

* fix typo in e2e test setup

* re-fix the e2e test setup

* be more tolerant of races

* anno matrix unit tests

* temp disable reembedding

* pilot port continuous histo to react-async

* name change

* lint

* fix repaint bug

* typo fix

* update snap to match new ids

* world/universe name cleanup

* move annoMatrix to src dir

* use private underscore naming convention

* fix corner case in all selected

* name cleanup

* add layout control

* init edge case

* lint

* port scatterplot

* fix label indexing bug and improve tests

* port category to react-async

* fix user annotation labelling while subset

* select all of prev layout on layout switch

* fix race with crossfilter update

* prettier lint

* fix misleading comment

* fix url composition in loader

* first pass at crossfilter tests

* lint

* lint

* fix typo

* improved error handling for network errors

* fix memoization bug

* add memo

* refactor for performnce

* add missing single-value handling in select exact parser

* small bugs discovered by tests

* lint

* additional crossfilter unit tests

* remove extraneous comment

* add support for automatic category determination

* lint

* fix render bug in category

* take advantage of schema categories guarantee

* lint

* do not clear history when resetting

* enhanced annomatrix gc

* lint

* finish renaming to follow conventions; fix clone race bug

* lint

* add priority based loading to improve initial data load UX

* crossfilter cache perf

* perf tuning

* remove timers

* documentation

* PR review changes

* PR review changes

* more PR review edits

* improve clarity of comment

* more PR review fixes

* port centroidLabels to use react-async

* remove dead code

* pr review updates

* oops, remove logging
2020-07-14 13:53:33 -07:00
Timmy Huang
2b0c4692ba e2e-add-more-logs (#1598) 2020-06-30 21:27:38 -07:00
Bruce Martin
8898a9817c protect against non-integer array indexing (#1595)
* do not assume non-iteger array bracket access will return undefined

* revert to spec
2020-06-29 09:37:59 -07:00
Timmy Huang
b79f09c550 FIX-test-target-closed: remove jestPuppeteer.resetBrowser() (#1592) 2020-06-26 10:02:25 -07:00
Timmy Huang
83376627e8 1510-smoke-test (#1548)
* 1510-smoke-test

* config default

* update tests

* update test config

* fix linter errors

* more comments

* address comments

* use npm install in push_tests.yml

* use environment.default.json

* adding docs

* Take care of @mweiden's nits

* Save screenshots in the __tests__/screenshots/ directory

* typo

* docs

* Add chart tests (#1580)

* merge tests

* check if bin creation returned null before rendering charts (#1576)

* check if bin creation returned null before rendering charts

* refactor chart rendering into functions (#1577)

* little fixes from PR

* reintroduce fix to check for null values

* change getAllByClass to return element

* slice instead

* new stackedbar test

* feedback-1573-test (#1579)

* feedback-1573-test

* enable whole test set

* revert tests

Co-authored-by: Timmy Huang <tihuan@users.noreply.github.com>

* tweak test to actually render chart

* include snapshot

* remove async

* fix getAllHistograms

* properly grab id

Co-authored-by: Timmy Huang <tihuan@users.noreply.github.com>

Co-authored-by: Matt Weiden <538456+mweiden@users.noreply.github.com>
Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com>
2020-06-24 11:45:39 -07:00
Severiano Badajoz
0e28df0bd6 Fix Truncation styling issues (#1558)
* fix color inheritance

* tweak width constants

* update snapshot

* Trigger workflow
2020-06-16 10:48:28 -07:00
Bruce Martin
bc7a712048 refactor categorical controls state (#1549)
* refactor categorical controls state

* lint

* fix race condition in tests

* fix typo

* add missing update on subset

* remove obsolete code

* update jest and puppeteer major version; update all minors

* update when label changes

* remove lint from tests; increase timeouts in e2e tests

* changes in response to PR review

* lint

* more PR comment changes

* more PR comment fixes

* lint

* more PR comment resolutions
2020-06-10 12:56:38 -07:00
Severiano Badajoz
4e96847032 Create Truncation Component (#1500)
* remove ESLint rules

operator-linebreak is no different then default
quotes is overwriting prettier config

* add string-pixel-width dep

* don't lint-staged src, only staged

* add widthMap

* create Truncate component

* refactor in truncate component

* add font load checking

* remove font-family styling

* render Truncate's child instead of creating own component to render

* refactor to use Truncate component

* add span back

* support children

* remove maybeTruncateString

* sub in Truncate component

* add bold prop

* accurately compute largest possible string

* remove logs

* tweak truncation method

* memoize comp function

* explain disable

* fix bugs w/ abs/floor

* tweak widths

* tweak widths

* fix font size

* remove border

* move test-id

* attempt css solution

* Revert "attempt css solution"

This reverts commit aac4d8a6f6.

* CSS solution v2

* remove string-pixel-width

* remove widthsMap

* remove dead code

* remove "data-truncated" as it is always true

* tweak label width

* fix e2e tests

* remove testing string

* e2e annotations tweaks

* correct snapshot

* remove resolves

* check for labels

* add test-id

* format fix

* update snapshot

* fix color

* pull constants out where available
2020-06-05 10:14:42 -07:00
Bruce Martin
2ba4944f5c initial bug fixes and test improvements for the matrix refactor (#1503)
* initial bug fixes and test improvements for the matrix refactor

* lint
2020-06-02 09:47:40 -07:00
Matt Weiden
8d762e59ca Deduplicate merged gene list in subsetAndResetGeneLists (#1490)
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1486
2020-05-20 15:35:27 -07:00
Matt Weiden
efdae22fbe Lengthen e2e test timeout to account for page load (#1488) 2020-05-20 14:55:43 -07:00
Severiano Badajoz
b06b08ce33 Fix centroid label + continuous color-by interactions (#1484)
* add case to removing labels

* add disabled state to centroid button

* fix centroid test button clicking order

* don't accidentally un-toggle first color
2020-05-20 14:02:10 -07:00
Severiano Badajoz
c34a68304e remove all linting errors on client/src (#1463)
* run eslint --fix

* camelcase

* camelCase config part 1

* part 2

* part 3 - removing subscripts

* fix "class-methods-use-this"

* fix "class-methods-use-this"

* fix eslint ignores

* add eslint ignore for set state in update

* reformat comments to appease eslint

* add a11y features

* sort-comp fix

* a11y fix

* add ignore for set state in update

* add a11y htmlFor

* remove unused toast

* remove unnecessary bind

* add ignore for set state in update

* add rel="noopener noreferrer"

Using target="_blank" without rel="noopener noreferrer" is a security risk: see https://mathiasbynens.github.io/rel-noopener

* use arrow function to bind

* remove unused definitions/declarations

* prettier

* remove unused state

* add comments to empty catch blocks remove curly brackets

* escape '

* use eqeqeq

* switch from default export

* remove ignore log

* remove static

* fix import

* revert subscripting config

* clean-up

* remove unnecessary subscript

* fix new errors from master

* change category click handler to a class property

* fix camelcase changes that slipped by

* unused import

* Fix newly introduced ESLint errors from addGenes
2020-05-19 12:38:29 -07:00
Matt Weiden
e55595cc55 Fix smoke tests to work with remote deployments of cellxgene (#1469)
* Add test for terms of service

* Add workaround for chromium CSP require-trusted-types-for error
2020-05-12 13:01:57 -07:00
Matt Weiden
888f730526 Refactor CSS and React logic for layout (#1424)
* Refactor CSS layout and react logic for layout

Fixes https://github.com/chanzuckerberg/cellxgene/issues/1022

* Menubar should wrap inside middle pane instead of overlapping left
  sidebar when window is scrunched
* cellxgene should have a minimum width of 1240px

1. Replace absolute positioning and dimension calculation with css grid
2. Use flexbox for wrapping menubar buttons

* Middle pane (graph) can calculate its own size
* Removing components calculating their size/position relative to
  eachother increases modularity, decreases use of global variables
* Improved some scrollbar behavior
* Removed responsive reducer, propagating window size to components
  triggers unnecessary events and encourages breaking modularity; doing
  this made some components state agnostic

Reference: https://css-tricks.com/snippets/css/complete-guide-grid/

* Reposition the continuous legend

* Small fixes

* Respond to feedback from @colinmegill

* Respond to feedback from @colinmegill

Add more documentation on the renderGraph method.
2020-05-06 13:46:13 -07:00
Severiano Badajoz
6cccc41c0f run prettier(2.0.5) (#1438) 2020-05-04 10:26:42 -07:00
Matt Weiden
a2047b90ae Make the URL hit by jest smoke tests configurable (#1420)
* Make the URL hit by jest smoke tests configurable

* Fix typo

* Make names consistent
2020-04-22 15:23:11 -07:00
Matt Weiden
1f94c71c22 Use sci notation for axis ticks abs(tick) >= 10000 (#1363)
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1349

For more information see https://github.com/d3/d3-format

Note that does not _fully_ fix the issue described in #1349, but rather
makes the formatting issue far less likely. It is _still_ possible for
this to occur if the difference between two ticks in axes happes in the
a significant digit cropped by the scientific notation format
2020-04-08 09:45:32 -07:00
Severiano Badajoz
4efabf5231 refactor to remove non-standard terms in centroid files (#1330)
* refactor to remove non-standard terms

* switch key/value -> label/coords

* fix missing parenth
2020-04-06 10:20:11 -07:00
Bruce Martin
708a5af039 Add fetch concurrency limit on obs annotation loading (#1318)
* add fetch concurrency limit

* add tests for PromiseLimit
2020-03-30 16:23:09 -07:00
Colin Megill
234f25b782 Conditionally truncate category string (#1206)
* maybe truncate string

* add string formatting to test

* correct import

* destructuring

* add maxlength

* test

* Respond to feedback from @bkmartinjr

Co-authored-by: Matt Weiden <538456+mweiden@users.noreply.github.com>
2020-03-10 18:14:26 -07:00
Severiano Badajoz
023a73cab8 smoke-test centroids + graph overlay layer (#1189)
* add test labels

* prettier + add clcik return to clickOn()

* prettier + begin test

* finish label counting test

* add util to get coordinates of element

* add test id to labels

* add test to check overlay transform

* remove logs

* rename to match master
2020-03-09 17:01:22 -07:00
Bruce Martin
144b19c449 experimental re-embedding (#1186)
* first cut at re-embedding route and back-end support

* update and expand config route tests

* add scanpy_umap

* add reembedding to config route parameters

* front-end support for reembedding fetch and UI

* remove unused imports

* add loading state

* save reembedding in reducer state

* improve withColsFrom

* transmit reembed schema to client; pick unique embedding names

* display embeddings

* format

* lint

* spaces, tab size 2

* lint

* test hack for smoke-test race

* back out hack sleep

* add check for backed mode

* add unit test for reembedding

* lint

* hide re-embedding CLI param from help
2020-03-09 16:53:30 -07:00
Matt Weiden
7b77bf4bdd Make smoke tests faster, more stable (#1195)
* Refactor smoke tests & utils for conciseness/style

* Modularize test utilities
2020-03-04 16:11:32 -08:00
Matt Weiden
05323ae643 Undo selection appends genes from differential expression to user gene list (#1183)
* Undo selection appends diffExp genes to user gene list

Fixes https://github.com/chanzuckerberg/cellxgene/issues/1171

Need: When a user performs a differential expression from within
a sub-selection (world) of the data and then resets the selection to all
cells (universe), the differential expression results are no longer
valid.

Approach:
* When the selection is reset, move the top (maxUserDefinedGenes
- len(userDefinedGenes) from the differential expression results to the
list of user defined genes
* Raise maxUserDefinedGenes to 25 to give users more room and
accommodate the extra genes transferred in from differential expression

Other commits:
* Choose different button icons
* Add diff exp genes to user defined genes on subset too
* Respond to feedback from @liaprins-czi and @bkmartinjr
2020-03-02 12:07:12 -08:00
Severiano Badajoz
0043308b52 add centroid util unit tests (#1172)
* scaffold the test

* create tests
2020-02-26 12:16:13 -08:00
Matt Weiden
8a62c9f027 Add undo/redo smoke tests for annotations (#1175)
* Add undo/redo tests for annotations

Fixes https://github.com/chanzuckerberg/cellxgene/issues/969

... also refactor the tests for DRY.

* Add done()

* Make e2e annotations tests safer to concurrency

* Add data-testclass for save state.

* Simplify tests and make them dependent on save state
2020-02-26 12:10:03 -08:00
Bruce Martin
fb1f0c6469 fix latent crossfilter bug (#1162) 2020-02-21 14:14:41 -07:00
Matt Weiden
d30087ca63 Add frontend smoke tests for annotations (#1154)
* Add smoke test for annotations features

* Do not save during annotations tests

* Fix botched rebase in dev guidelines

* Revert "Do not save during annotations tests"

This reverts commit f0bd970bb2.

* Respond to feedback from @bkmartinjr
2020-02-14 17:23:45 -08:00
Matt Weiden
ac13b31e13 Collect all env vars in one, easy-to-find place (#1149)
* Collect all env vars in one, easy-to-find place

Past state:
* Default environement variables were stored in both client/package.json
and client/__tests__/e2e/config.js
* Constants that should have been linked--like the cellxgene server port
during testing--were repeated.

With this commit:
* All environment variables are parameterized
* All environment variables are packaged in default env files

* Move npm scripts to client Makefile

* Respond to feedback from @seve and @bkmartinjr
2020-02-12 12:50:48 -08:00
Bruce Martin
e770db1e2c load annotations incrementally (#1107)
* load annotations individually

* fix type check to be more general

* update node CI version from 10 to 12

* node 11

* debug print node version

* travis node version to latest

* try nvm

* remove extraneous node_js statement

* remove node version debugging printf

* incrementally load all annotations and layout

* process annotations and layout as they are loaded

* fix tests

* sort categories incrementally

* incrementally build category view summary; add category loading spinner

* add spinner to continuous metadata

* configure undoable reducer

* incremental crossfilter creation

* improve busy layout

* more layout cleanup

* correctly reconcile categories in schema

* refine layout of lsb spinners

* more spinner layout work

* more spinner layout

* always load layout before obs annotations
2020-02-10 11:22:27 -08:00
Matt Weiden
ddbfc458d7 Remove d3 dependency in crossfilter (#1108)
* Remove d3 dependency in crossfilter

Fixes https://github.com/chanzuckerberg/cellxgene/issues/648

* Revert "Remove d3 dependency in crossfilter"

This reverts commit 03b478f7dc.

* Remove d3 dependency in crossfilter

Fixes https://github.com/chanzuckerberg/cellxgene/issues/648

* Fix typo in docstring
2020-01-14 13:46:53 -08:00
Matt Weiden
4c1ecf9e52 Specify histogram thresholds in fixed-length array (#1086)
* Specify histogram thresholds in fixed-length array

Fixes https://github.com/chanzuckerberg/cellxgene/issues/1082

For background, see the following:
* https://github.com/d3/d3-array/issues/46
* https://stackoverflow.com/questions/15880058/d3-js-ticks-function-giving-more-elements-than-needed

* Oops, off by one!
2020-01-06 11:13:02 -08:00
Matt Weiden
b172ff7f15 Add margins to histogram SVGs (#1085)
* Add a left margin to histogram SVGs

Fixes https://github.com/chanzuckerberg/cellxgene/issues/1042

1. Added "margins" to the SVG as in https://bl.ocks.org/mbostock/3019563
2. Tailored the brushable area to be just where the data is and locked
   puppeteerUtils to use that in e2e testing.

* Dial in margins to avoid y-axis clipping
2020-01-06 10:46:06 -08:00
Colin Megill
4c3d783fc8 Continuous data in right sidebar (#985)
* Create right sidebar files, move histos over

* restructure continuous legend + all padding

* right sidebar file

* proper graph positioning and resize

* new lasso values since right sidebar

* fix autosave position
2019-11-05 20:29:21 -05:00
Bruce Martin
a16c2e2747 crossfilter immutability error (#1009)
* add rename immutability test

* make rename immutable
2019-11-04 14:14:17 -08:00
Bruce Martin
3f2811d9da rendering performance improvements (#968)
* freeze objects

* component rendering perf work

* use PureComponent where safe

* remove obsolete WorldUtil code

* make brushable histogram a pure component
2019-10-04 07:59:22 -07:00
Bruce Martin
3660a6cc27 Experimental - manual annotations (#837)
* icons, partway

* redux for values

* onChange

* cancel

* annotations lifecycle for category names

* copy categorical

* edit category

* add Dataframe.withColsFrom

* render user annotations; default add/delete annotation category

* add label name to actions

* category name edit

* error checking improvements

* change schema field isUserAnnotation to writable

* always have an unassigned label; implement delete label

* implement add new label and edit label name

* label current cell selection

* fix select exact bug in crossfilter

* clean up categorical reducer

* fix tests

* remove debugging printf

* implement subset/reset for user annotations

* undo redo support for user annotations

* remove duplicate button from categories

* add modal

* remove obsolete duplicate annotation reducers

* remove old debugging printf

* connect modal to annotation create and dup

* initial full-stack wiring

* finish up end-to-end wiring

* fix existing unit tests

* fix pytests to match new schema API

* remove debugging printfs

* add label file rotation

* remove obsolete comment

* add fbs encode/decode tests

* add tests for writable annotations

* simplify code

* fix hashing bug with FBS encoding

* lint

* fix smoke tests

* improve error checking in Dataframe.withColsFrom

* add unit test for Dataframe.withColsFrom

* add unit test for Dataframe.columns and Dataframe.renameCol

* fix bug in FBS encode, add better error checks, refactor

* add FBS encode/decode test

* add clarifying comment

* clean up action type names; fix state inconsistency in crossfilter update

* change autosave timer to 2.5sec

* sort categorical metadata render order so it remains consistent

* add temporary autogenerated label for add-new-label operation

* fix hover-over label menu interference with cell highlighting

* remove debugging code

* add missing reducer cases & fix typo

* make dataframe memoize more general purpose

* add dev mode for annos

* fix error on select duplicate

* handle zero occupancy categories

* correctly maintain unclipped AND clipped world

* correctly handle zero length FBS matrix and label files

* ensure all writable categorical schema contains an unassigned category

* handle case where building occupancy stack for category with no members

* dialog for creating label, disable button if duplicate or empty

* visually separate writeable

* edit category

* fix edit category name

* remove debugging code

* fix edit annotation label

* visually define unassigned, change options

* Pull in requirements.txt from `master`

* label currently selected cells

* duplicate label

* lint

* fix pytest merge issues

* rename --label-file to --experimental-label-file

* remove debugging console log

* spelling error fix; fix bug found in PR review.

* lint
2019-09-18 07:33:41 -04:00
Bruce Martin
0f36b49840 Graph/scatterplot improvements (#893)
* many graph/scatterplot fixes: 722, 882, 758, 886

* fix e2e test breakage caused by graph transform work

* convert webgl point shaders to use flags rather than explicit point size

* remove unused packages from dependencies

* fix typo in regl prop name

* factor common code into util helper file

* legibility tweaks
2019-08-22 20:56:05 -07:00
Bruce Martin
906d65c06f fix binning error in continuous data histogram (#869) 2019-08-05 10:40:56 -07:00
Severiano Badajoz
941c297363 categorical vs continuous mini histograms (#827)
* comment

* add histogram functionality to Dataframe; port category occupancy to use it

* fix binning and create histogram for continous by catagorical

* Remove unnecessary logs

* Begin work on KDE

* Replace broken KDE with working histogram

* Define domain and range based on data from histogram

* Fix occupancy

* Add continuous obs and switch to canvas

* Stop value from always rerendering

* clear before render

* Clear canvas on render

* refactor categorical occupancy to canvas

* Remove log

* simplify finding max

* refactor kde->histogram and occupancy->bins

* refactor svg -> canvas

* rename to occupancy stack

* create popup

* add metadata and categorical values to popup

* fix overflow

* remove zeros info

* style graph

* fix shouldComponentUpdate to look for world changes

* change categorySelected -> categoryValueSelected

* refactor out render

* remove comment

* conditionally have bottom border

* remove diff comp

* remove comments

* remove unnecessary mapping

* Add comments describing drawing functions

* comments

* flip comparison order

* remove logging

* move default to parameter

* move defaults to parameter

* disable popover if not showing histogram

* fix wording and styling

* add line break
2019-07-09 11:19:01 -07:00
Colin Megill
6aeefb0fe6 menubar (#804)
* menubar 1

* zoom switching

* centering, pixel perfect canvas

* remove dead args and code

* clipping

* remove log

* if

* connect props

* lint

* undo

* logo left, componetize

* graph back to full height

* shadow to top

* do not prematurely call event handlers during render

* change test to deal with async histogram creation

* left section padding

* lint

* adjust graph to account for top bar,

* lasso tests

* refine histogram tests

* remove testing (onlys)
2019-06-13 15:15:43 -07:00
Charlotte Weaver
1d6bb032a3 remove async from describe blocks (#805) 2019-06-03 14:59:16 -07:00
Bruce Martin
a6142bdf93 improve graph scale and centering (#796)
* add gutter to embedding canvas

* improve layout scale and translate

* fix lint

* pin tables to version 3.5.1

* fix lasso coordinate smoke tests
2019-05-30 13:31:41 -07:00
Bruce Martin
3dc45d6330 do not hard-wire column names in annotations (#785)
* enforce column name uniqueness for obs and var

* parameterize the column name containing obs and var user-readable names

* use the new annotation index value from schema

* update f/e unit tests

* PR review suggestions

* lint
2019-05-24 21:00:54 -07:00
Bruce Martin
e941c1a496 scaling omitted from event handlers (#789)
* scaling omitted from event handlers

* fix smoke tests
2019-05-24 07:00:04 -07:00
Colin Megill
a657eb3152 Logo (#782)
* logo, black

* fixes

* remove template, move header
2019-05-23 11:47:24 -04:00
Bruce Martin
2357d0c1b8 layout change UI (#776)
* add layout to schema

* add layout choice action and reducer

* multi layout UI

* update unit tests

* add missing file

* update test schema

* fix duplicate test id

* fix tabs

* PR lint

* fix pytest
2019-05-22 13:21:33 -07:00