Commit Graph
359 Commits
Author SHA1 Message Date
Matt Weiden 95ade476e8 Add end of line marker to lint diff regexp (#1400)
This prevents json files from being linted
2020-04-15 13:46:36 -07:00
Colin Megillandbkmartinjr ea434fc46f Handle loading state when rendering categories with one label (#1393)
* refactor & loading state

* truncate

* fix minor PR review issues

* lint

Co-authored-by: bkmartinjr <bruce@chanzuckerberg.com>
2020-04-15 14:38:44 -04:00
Matt Weiden 7d4d360e52 Refactor developer convenience scripts (#1377)
* Cleanup the backend-dev convenience method

* Add the frontend_dev convenience method

frontend_dev is a soup-to-nuts convenience method for setting up the FE
development environment with node running a the client code on port 3000
with the a separate cellxgene package serving the API over port 5005 in
the background.

The script can be run from Finder.

* Update the developer scripts documentation

* Remove the 'test' make target in the client Makefile

Rationale:
* Given how long the smoke tests take to run, it is unlikely that
  developers will want to run all tests together.
* It is unlikely that developers will have set up the backend server
  properly for the tests to pass.
* Available commands should be safe-ish and not lend themselves to
  confusing errors.
* You can still group tests by concatenating them in a make command, as
  in `make unit-test smoke-test`.
* This target isn't used in any of our CI pipelines -- KISS.

* Some version of python3...

* Minor typos in docs

* Respond to feedback from @bkmartinjr

* Make adjustments so that DATASET path is predictable

* Simplify environment defaults a bit
2020-04-15 10:32:19 -07:00
Severiano Badajoz e5dfd6e8fa Add eslint steps to lint job (#1335)
* add lint-client and lint-diff-client targets

* add lint-diff and lint targets

* prettier

* add lint-diff call to lint task

* temp

* tweak lint-diff

* add lint for PRs and lint for master

* remove temp

* remove incorrect branches syntax, use github_ref

* pull all branches

* format

* proper target and comment

* create separate steps with conditionals

* fix indentation

* refactor lint->lint-server, introduce lint to lint all

* trade diff-index for diff, do check against base instead of master

* remove fetching all branches

* Revert "remove fetching all branches"

This reverts commit 26ce7a0f05.

* tweak comparison

* use local eslint

* add eslint  dep install

* temp

* grab only base

* simplify fetch

* add pull_request type trigger

* specify pushes only to master

* change conditionals to be based on event name

* Revert "temp"

This reverts commit 3d59134cc0.

* "branch" => "branches"

* create separate installation step

* change command based on os
2020-04-15 09:41:45 -07:00
Colin Megill 64bd908c25 Render categories with one label without checkbox (#1375)
* categories with 1 item rendered without checkbox

* break out into helper functions

* prettier
2020-04-13 23:04:23 -04:00
Bruce Martin a3df1a9f4a performance and correctness fixes to brushable histogram (#1390) 2020-04-13 20:03:46 -07:00
Bruce Martin 136093d583 add GET routes for expression data (#1387)
* add GET routes for expression data

* fix comment typo
2020-04-13 11:28:35 -07:00
Bruce MartinandColin Megill e2a12ba9bb diffexp limit UI and configuration (#1336)
* warning on maxCount for diffexp

* cleanup logging

* clarification

* make the limits configurable

* make diff exp limit work

* danger!

* remove debugging code

* fix merge with master

* fix unit tests

Co-authored-by: Colin Megill <colinmegill@gmail.com>
2020-04-09 07:13:41 -07:00
Matt Weiden 1f94c71c22 Use sci notation for axis ticks abs(tick) >= 10000 (#1363)
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1349

For more information see https://github.com/d3/d3-format

Note that does not _fully_ fix the issue described in #1349, but rather
makes the formatting issue far less likely. It is _still_ possible for
this to occur if the difference between two ticks in axes happes in the
a significant digit cropped by the scientific notation format
2020-04-08 09:45:32 -07:00
Bruce Martin bcacb75296 load embeddings in parallel (#1352)
* load embeddings in parallel

* correctly capture unclipped

* test

* another test

* add convenient copy assets target

* cleanup
2020-04-07 09:48:07 -07:00
Bruce Martin 6193ae4997 Cleanup front-end build and python module contents (#1350)
* cleanup build and module contents

* lint

* update chalk

* more cleanup

* fix unit test
2020-04-06 19:04:06 -07:00
Severiano Badajoz 4efabf5231 refactor to remove non-standard terms in centroid files (#1330)
* refactor to remove non-standard terms

* switch key/value -> label/coords

* fix missing parenth
2020-04-06 10:20:11 -07:00
Bruce Martin de8033e551 correctly handle categoricals that are not strings (#1346) 2020-04-03 15:50:02 -07:00
Bruce Martin 7c56041b11 categorical type handling fix (#1342)
* fix numeric category conversion bug

* correctly compute categorical summaries

* lint

* remove debugging print
2020-04-02 14:44:07 -07:00
Bruce Martin 708a5af039 Add fetch concurrency limit on obs annotation loading (#1318)
* add fetch concurrency limit

* add tests for PromiseLimit
2020-03-30 16:23:09 -07:00
Severiano Badajoz 033727632c add truncation to loading (#1320) 2020-03-30 15:48:52 -07:00
Bruce MartinandColin Megill c8f98917c5 tos prompt (#1313)
* tos toast

* finish ToS prompt

Co-authored-by: Colin Megill <colinmegill@gmail.com>
2020-03-28 22:31:19 -07:00
Bruce Martin 57938b487b remove static route redirects / bump static to top level (#1299)
* add another favicon rule

* catch only top-level dataset names

* add config flag to disable redirects

* remove static redirect, point all links to top-level /static

* remove extraneous argument

* remove httpd redirect
2020-03-26 10:45:10 -07:00
Colin Megill adf7010f50 Add TOS and Privacy url via config (#1300)
* add tos and privacy url via config

* typo

* readd condition

* param
2020-03-26 11:33:41 -04:00
Bruce Martin 752b9e4ab3 CORS and CSP headers (#1286)
* do in-app compression only for CLI

* CORS and CSP headers

* lint

* add --debug to targets

* lint

* fix botched merge with master
2020-03-24 08:05:37 -07:00
Severiano Badajoz b249df9036 add/fix label tooltips (#1278)
* Move tooltip to right

* Change function to only return truncated string

* add tooltip to category label

* use position instead of PopoutPosition
2020-03-23 10:28:54 -07:00
Colin Megill c4731d52e2 target blank (#1254) 2020-03-20 20:07:51 -04:00
Bruce Martin 8f09a4b1bc JS package fixes (#1248)
* Remove accidental package install from build

* remove accidental package install
2020-03-18 19:00:28 -07:00
bmccandless 8a733270d7 fixes for hosted cellxgene (#1227)
* Fix for favicon with --dataroot

* fix static assets in hosted cxg 

The web proxy at aws eb was not finding the static assets.

The solution here is very simple: just copy the directory
containing the static assets to the top level of the artifact.zip.

This is not really the ideal solution.  According to the AWS
docs you can make a mapping to the correct location in an
an ebextentions config file.  I tried this and many combinations but
was not able to get this to work following that pattern.

Since we control the construction of the zip file, the solution
here isn't bad, but it could probably be made better.
2020-03-16 09:47:01 -07:00
Matt WeidenandDonald Paul Herman 86cbe64ae8 Add color mapping to the bar chart (#1218)
* Add color mapping to the bar chart.

* Change histogram color generation from Viridis to InterpolateCool.

* Coloring of Histrogram based up the x axis instead of y axis.

Coloring of Histrogram based up the x axis instead of y axis.

* Respond to feedback from @colinmegill

* Only color histograms that are selected for colorby
* Add some small refactors to BrushableHistogram's componentDidUpdate

* Fix histogram coloring and binning

* Reuse binning functions from util/dataframe/histogram.js; this fixes
an issue with there being near-zero width bins
* Fix color mapping so that it matches the scale in the legend
* Do not attempt to plot bins if the calculated binWidth is zero; this
can happen if all values are the same
* Refactor the function that draws the histogram a bit

* Respond to feedback from @bkmartinjr

Co-authored-by: Donald Paul Herman <Donaldpherman@hotmail.com>
2020-03-14 15:24:52 -07:00
Severiano Badajoz 9c2f57df39 bump version to 0.15.0 (#1222) 2020-03-13 13:11:12 -07:00
Bruce Martin f28bd52d79 fix performance regression caused by excessive rendering (#1219) 2020-03-12 15:05:08 -07:00
Bruce Martin 63f1f38f7c use AnchorButton so the tooltip dismisses (#1217) 2020-03-12 08:36:36 -07:00
Severiano Badajoz 3ba4cf4483 fix bug created from master rebase (#1216) 2020-03-11 15:20:20 -07:00
Severiano Badajoz 3bf72829c2 pass empty coordinates when a continuous annotation is colored (#1212) 2020-03-11 13:01:14 -07:00
Severiano Badajoz 7cd9a0032a optimize centroid util (#1147)
* move unvarying evaluations outside of loop

* refactoring

* remove perf checks

* commenting

* minor fix + renaming

* small fix

* benchmarking

* merge master

* Revert pref checks

* renaming and comment

* remove redundant sets in Map
2020-03-11 12:35:42 -07:00
Sidney BellandMatt Weiden 9089fc98f2 annotations cli updates (#1190)
* Switch logic from `--annotations` to `--disable-annotations

* Rename `--annotations-file` --> `--annotations-input-file` and remove `experimental`

* update docs

* update makefile

* update tests

* Disable annotations on standard client smoke test

* Update docs/posts/annotations.md

Co-Authored-By: Matt Weiden <538456+mweiden@users.noreply.github.com>

* Update docs/posts/annotations.md

* Renaming

* Docs

* Update tests

* Pesky typo -_-

Co-authored-by: Matt Weiden <538456+mweiden@users.noreply.github.com>
2020-03-11 11:57:52 -07:00
Colin MegillandMatt Weiden 234f25b782 Conditionally truncate category string (#1206)
* maybe truncate string

* add string formatting to test

* correct import

* destructuring

* add maxlength

* test

* Respond to feedback from @bkmartinjr

Co-authored-by: Matt Weiden <538456+mweiden@users.noreply.github.com>
2020-03-10 18:14:26 -07:00
Bruce Martin 28ddb60b2f update JS deps (#1203) 2020-03-10 10:59:12 -07:00
Colin Megill 1a93068810 Prevent category collapsing on annotation edit, # of cells on add label button (#1204)
* collapse functionality

* expand collapse
2020-03-10 13:21:06 -04:00
Severiano Badajoz 023a73cab8 smoke-test centroids + graph overlay layer (#1189)
* add test labels

* prettier + add clcik return to clickOn()

* prettier + begin test

* finish label counting test

* add util to get coordinates of element

* add test id to labels

* add test to check overlay transform

* remove logs

* rename to match master
2020-03-09 17:01:22 -07:00
Bruce Martin 144b19c449 experimental re-embedding (#1186)
* first cut at re-embedding route and back-end support

* update and expand config route tests

* add scanpy_umap

* add reembedding to config route parameters

* front-end support for reembedding fetch and UI

* remove unused imports

* add loading state

* save reembedding in reducer state

* improve withColsFrom

* transmit reembed schema to client; pick unique embedding names

* display embeddings

* format

* lint

* spaces, tab size 2

* lint

* test hack for smoke-test race

* back out hack sleep

* add check for backed mode

* add unit test for reembedding

* lint

* hide re-embedding CLI param from help
2020-03-09 16:53:30 -07:00
Matt Weiden bdd69160e2 Do not over-prune var data cache (#1198)
* Remove unused variables and imports

* Simplify conditional

* Fix typo

* Do not overprune var data cache

There is a bug in how the universe and world gene sets are constructed
and passed to `ControlsHelpers.pruneVarDataCache` that causes the var
data cache to be over-pruned. This commit fixes the issue.

Consider the following example from the node console:

```
❯ node
Welcome to Node.js v13.5.0.
Type ".help" for more information.
> new Set([1], [2], [3])
Set(1) { 1 }
```

What we really want is the set `Set(3) { 1, 2, 3 }`, which can be
constructed as:

```
> new Set([].concat([1], [2], [3]))
Set(3) { 1, 2, 3 }
```
2020-03-05 18:04:14 -08:00
Matt Weiden bddf66d62a Use blueprint "disable" property to indicate disabled buttons (#1197)
* Use blueprint "disable" property to indicate that colorby disabled

* Disable subset and reset buttons when action is not possible
2020-03-05 09:46:41 -08:00
Matt Weiden 7b77bf4bdd Make smoke tests faster, more stable (#1195)
* Refactor smoke tests & utils for conciseness/style

* Modularize test utilities
2020-03-04 16:11:32 -08:00
Severiano Badajoz 62e944796b enable camera interaction on centroids (#1160)
* pass handleCanvasEvent to graphOverlayLayer

* dont pass handlecanvasevent if not interactive
2020-03-04 11:31:30 -08:00
Matt Weiden 1547a864c2 Disable ColorBy button for truncated categories (#1191)
* Disable ColorBy button for truncated categories

Fixes https://github.com/chanzuckerberg/cellxgene/issues/1156

For categories that have more than 100 labels we truncate the labels in
the UI, but still allowed users to ColorBy these categories. Coloring by
these categories can cause browsers to get bogged down.

This commit disables ColorBy for truncated categories.

* Minor documentation spelling and typo fixes

* Respond to feedback from @liaprins-czi

* Respond to feedback from @colinmegill and @bkmartinjr
2020-03-04 09:19:12 -08:00
Matt Weiden 05323ae643 Undo selection appends genes from differential expression to user gene list (#1183)
* Undo selection appends diffExp genes to user gene list

Fixes https://github.com/chanzuckerberg/cellxgene/issues/1171

Need: When a user performs a differential expression from within
a sub-selection (world) of the data and then resets the selection to all
cells (universe), the differential expression results are no longer
valid.

Approach:
* When the selection is reset, move the top (maxUserDefinedGenes
- len(userDefinedGenes) from the differential expression results to the
list of user defined genes
* Raise maxUserDefinedGenes to 25 to give users more room and
accommodate the extra genes transferred in from differential expression

Other commits:
* Choose different button icons
* Add diff exp genes to user defined genes on subset too
* Respond to feedback from @liaprins-czi and @bkmartinjr
2020-03-02 12:07:12 -08:00
Bruce Martin bf7d7342d5 Improve label picking (#1179)
* add simple error message helper

* port all label name pickers to use the new LabelInput component

* use pure components where possible

* cleanup

* more cleanup

* lint

* change new label prompt
2020-02-28 15:46:40 -07:00
Matt Weiden 1e4381ab7f Make the Clip button primary (blue) (#1181)
Fixes https://github.com/chanzuckerberg/cellxgene/issues/1101

I can't reproduce the active/inactive issue described in #1101

The only change remaining in #1101 is to make the clip button blue.
This commit makes that change.
2020-02-26 16:54:30 -08:00
Severiano Badajoz 0043308b52 add centroid util unit tests (#1172)
* scaffold the test

* create tests
2020-02-26 12:16:13 -08:00
Matt Weiden 8a62c9f027 Add undo/redo smoke tests for annotations (#1175)
* Add undo/redo tests for annotations

Fixes https://github.com/chanzuckerberg/cellxgene/issues/969

... also refactor the tests for DRY.

* Add done()

* Make e2e annotations tests safer to concurrency

* Add data-testclass for save state.

* Simplify tests and make them dependent on save state
2020-02-26 12:10:03 -08:00
Bruce Martin 7bc58bba2b undo/redo cleanup (#1165)
* fix refactoring error which disabled annotation file clearing

* fix undo behavior on add category label

* fix various undo/redo bugs

* remove logging

* further refinement of annotation undo/redo and actions

* address PR comment
2020-02-24 18:51:26 -07:00
Matt Weiden 5c70cc5bcd Add codecov for code coverage of python and javascript (#1170)
* Add codecov to Push Test workflow

* Empty commit

* Clear reports and tag each with flags

* Tag code reports by test

* Fix codecov tags

* One more fix
2020-02-24 17:13:16 -08:00
Matt Weiden c7f2032dd7 Add user-generated annotations tests to the server (#1164)
* Add user-generated annotations tests to the server

Partially completes https://github.com/chanzuckerberg/cellxgene/issues/969

* Auto-format python code

* @skip_if: passing lambdas > than property strings

* Respond to feedback from @bkmartinjr
2020-02-23 15:32:13 -08:00