Commit Graph
100 Commits
Author SHA1 Message Date
Bruce Martin ffdcc24a40 remove pinned tables dependency (#1202) 2020-03-10 07:47:54 -07:00
Bruce Martin 144b19c449 experimental re-embedding (#1186)
* first cut at re-embedding route and back-end support

* update and expand config route tests

* add scanpy_umap

* add reembedding to config route parameters

* front-end support for reembedding fetch and UI

* remove unused imports

* add loading state

* save reembedding in reducer state

* improve withColsFrom

* transmit reembed schema to client; pick unique embedding names

* display embeddings

* format

* lint

* spaces, tab size 2

* lint

* test hack for smoke-test race

* back out hack sleep

* add check for backed mode

* add unit test for reembedding

* lint

* hide re-embedding CLI param from help
2020-03-09 16:53:30 -07:00
Bruce Martin bf7d7342d5 Improve label picking (#1179)
* add simple error message helper

* port all label name pickers to use the new LabelInput component

* use pure components where possible

* cleanup

* more cleanup

* lint

* change new label prompt
2020-02-28 15:46:40 -07:00
Bruce Martin 7bc58bba2b undo/redo cleanup (#1165)
* fix refactoring error which disabled annotation file clearing

* fix undo behavior on add category label

* fix various undo/redo bugs

* remove logging

* further refinement of annotation undo/redo and actions

* address PR comment
2020-02-24 18:51:26 -07:00
Bruce Martin fb1f0c6469 fix latent crossfilter bug (#1162) 2020-02-21 14:14:41 -07:00
Bruce Martin d2bae0c4f6 CXG creator and dataset parameters (#1163)
* remove redundant call to make_response (lint)

* initial commit of H5AD to CXG converter

* lint

* PR review suggested comments

* cxg adaptor now understands group metadata

* lint
2020-02-21 12:49:45 -07:00
Bruce Martin 349c413d8b anndata X indexing & version compatibility improvements (#1157)
* revert MatrixProxy; replace with correct use of adata slicing

* work around 0.6 adata slicing bug

* fix incorrect var slice

* simplify slicing of X

* add warning about performance impact of anndata<=0.7

* lint and remove unused code

* improve comment

* lint

* correctly parse versions

* temp files should preserve file suffix if possible - anndata 0.7 compat

* update anndata dependency to 0.6.20

* resolve PR review comments
2020-02-19 09:57:51 -07:00
Bruce Martin e770db1e2c load annotations incrementally (#1107)
* load annotations individually

* fix type check to be more general

* update node CI version from 10 to 12

* node 11

* debug print node version

* travis node version to latest

* try nvm

* remove extraneous node_js statement

* remove node version debugging printf

* incrementally load all annotations and layout

* process annotations and layout as they are loaded

* fix tests

* sort categories incrementally

* incrementally build category view summary; add category loading spinner

* add spinner to continuous metadata

* configure undoable reducer

* incremental crossfilter creation

* improve busy layout

* more layout cleanup

* correctly reconcile categories in schema

* refine layout of lsb spinners

* more spinner layout work

* more spinner layout

* always load layout before obs annotations
2020-02-10 11:22:27 -08:00
Bruce Martin 1c9b9f6a08 js package dep update (#1145)
* js package dep update

* roll back start-server-and-test
2020-02-05 13:11:05 -08:00
Bruce Martin 3afc03b95b fix state prop bug with camera transform (#1138) 2020-01-31 12:13:35 -08:00
Bruce Martin 69c639b14d pin anndata to 0.6.22post1 (#1122) 2020-01-22 13:23:05 -08:00
Bruce Martin 431c8bdabc disable magic nan parsing in read_csv (#1115) 2020-01-17 09:26:15 -08:00
Bruce Martin 8b329630f8 fix for issue 1112 - all user annotation category names must be unique across all obs annotatoins (#1114) 2020-01-16 09:28:29 -08:00
Bruce Martin 30f3ce49e4 update JS dependencies (#1106) 2020-01-15 17:11:51 -08:00
Bruce Martin 677433bbf5 Fix label sorting bugs (#1102)
* move category label sort to utils

* refactor cat label sort

* fix category label sort and color assignment

* convert whitespace from tabs to spaces
2020-01-09 13:34:25 -08:00
Bruce Martin 2d39cb3b30 sort the embedding names for the UI (#1098) 2020-01-09 06:33:14 -08:00
Bruce Martin 18177b108f fix 1090 - add unassigned label to the new category, not the source category (#1094) 2020-01-09 06:32:38 -08:00
Bruce Martin dfaf0d0603 allow hyphen in annotation file name (#1074)
* remove old detritus

* allow hyphen in label file name
2020-01-02 15:44:40 -08:00
Bruce Martin 6370025caf update all JS package deps (#1076)
* update dependencies - minor revisions

* update min node_js version to 10

* major npm package dep updates

* a few more updates
2020-01-02 09:14:09 -08:00
Bruce Martin c9b6044c4f release 0.13.0 (#1064) 2019-11-26 15:45:13 -08:00
Bruce Martin fa89a5a375 create annotations output dir if it does not exist (#1062)
* create annotations output dir if it doees not exist

* lint
2019-11-26 13:35:20 -08:00
Bruce Martin f8cef71eed help text changes (#1061)
* requested CLI help changes for launch sub-command

* update file name prose

* more wording changes

* more PR requests
2019-11-26 13:35:01 -08:00
Bruce Martin d2171b9a2f fix perf regression introduced by user anno changes (#1060) 2019-11-25 18:16:49 -08:00
Bruce Martin a593e95ab3 annotations CLI and file UX rework (#1049)
* rename config param label-file

* annotations rework - CLI params, file naming and backups

* lint

* improve cli option error checks

* enable session cookies

* enable session cookies

* add session id

* name annotations file in multi-dataset and multi-user safe manner

* pass data user hash to front-end

* add annotation collection name support to front-end

* add constant for annotation data collection name

* parameterize annotation collection name; make it sticky in the session

* clarify comments

* hard wire a temporary data collection name for testing

* prettier

* test comment

* package command

* set annotations  filename dialog

* name  and hash are visible

* wire up data collection capture
2019-11-25 15:28:28 -08:00
Bruce Martin 5713aa8792 category and label name validation updates (#1056)
* update label validation with new schema

* add new error messages for new label errors; general cleanup
2019-11-22 10:32:00 -08:00
Bruce Martin ee62dd355f [WIP] JS lint and dead code removal (#1053)
* lint and dead code removal

* fix regressions
2019-11-21 14:10:02 -08:00
bkmartinjr 0415c30f0b add security notice to readme 2019-11-20 10:42:27 -08:00
bkmartinjr 79e534e397 additional caveats/warnings for hosting 2019-11-20 10:40:00 -08:00
bkmartinjr f4919340e9 merge 2019-11-20 10:33:17 -08:00
bkmartinjr 976f9d9ff8 add hosting notes 2019-11-20 10:28:40 -08:00
Bruce Martin 9a540a4f0a fix erroneous state clearing (#1052) 2019-11-19 11:11:19 -08:00
Bruce Martin 2d0b646956 js dependency update (#1045) 2019-11-18 11:22:13 -08:00
Bruce Martin b284e6f820 Improve CLI help (#1025)
* launch option changes

* more CLI help improvements

* change plot help

* additional changes requested

* change metavars for options and subcommand
2019-11-14 13:02:40 -08:00
Bruce Martin 5ea9ac8a9e Add provenance header to labels CSV file (#1041)
* add last mod time lookup to data locator

* store data locator in Driver

* save metadata header in labels csv

* lint

* fix tests

* change datetime format to second precision
2019-11-14 12:33:48 -08:00
Bruce Martin a52e86a69b improve category and label name validation (#1034) 2019-11-13 09:40:37 -08:00
Bruce Martin 04c83993d2 clear color-by mode when current category is deleted (#1033) 2019-11-12 15:31:06 -08:00
bkmartinjr 54d144bb75 add build dependencies for igraph and louvain 2019-11-12 07:56:03 -08:00
bkmartinjr bf3bf58d6e improve launch docs 2019-11-09 13:03:47 -08:00
bkmartinjr 611c8b98b1 add --backed info to troubleshooting 2019-11-09 12:59:33 -08:00
bkmartinjr d08506ae12 improve install docs 2019-11-09 12:49:22 -08:00
Bruce Martin 238b7420f7 update deprecated Pandas API (#1021)
* update use of deprecated API

* update pandas depencency
2019-11-08 09:49:52 -08:00
Bruce Martin 461084eca1 Pin the 'unassigned' annotation value/label to bottom of list (#1018)
* pin unassigned label to bottom of category list

* pin unassigned label to bottom of category list
2019-11-05 09:22:39 -08:00
Bruce Martin 341d3015f0 fix typo (#1016) 2019-11-05 09:21:48 -08:00
Bruce Martin 7599af451f add context sensitivity to annotation "label" menu item (#1015)
* cache common results to reduce jank

* add util function in support of annotation labelling

* enable/disable annotation edit menus per issue 972; improve formatting

* PR review requests
2019-11-04 14:22:38 -08:00
Bruce Martin 823d819eee allow user to save an empty label dataframe (#1012) 2019-11-04 14:14:49 -08:00
Bruce Martin a16c2e2747 crossfilter immutability error (#1009)
* add rename immutability test

* make rename immutable
2019-11-04 14:14:17 -08:00
Bruce Martin 60a23b681c QOL documentation: csweaver contributions from PR #912 (#977)
* csweaver changes from PR #912

* fix formatting error
2019-10-24 08:44:36 -07:00
Bruce Martin e581dec8f5 release 0.12.0 (#981) 2019-10-08 12:34:21 -07:00
Bruce Martin 711f3b7048 Add support for anndata backed mode (#943)
* initial cut at backed mode

* make flask multithreading conditional on debug flag

* update X access to support backed mode

* lint

* improve help message for backed mode

* fix tests

* add MatrixProxy to normalize supported matrix types

* add FAQ entry for --backed

* remove use of matrix.T

* clean up

* add ability to disable diffexp from CLI; add hueristic to detect likely slow diffexp calculation, and warn user

* fix tests

* do not print diffexp speed warning if diffexp is disabled

* tweak wording of diffexp speed messages

* add FAQ entry on --disable-diffexp

* revise heuristic for warning about slow diffexp

* use quick tooltip delay on diffexp button
2019-10-08 11:16:07 -07:00
Bruce Martin de6721b682 fix broken link (#978) 2019-10-07 11:08:18 -07:00
Bruce Martin 3f2811d9da rendering performance improvements (#968)
* freeze objects

* component rendering perf work

* use PureComponent where safe

* remove obsolete WorldUtil code

* make brushable histogram a pure component
2019-10-04 07:59:22 -07:00
Bruce Martin 700c871e6d reduce dot scaling upon zoom (#963)
* remove obsolete code

* change point scaling with zoom distance
2019-10-04 07:58:38 -07:00
Bruce Martin ad290b3630 scale scatterplot point rendering size with overall app size, not scatterplot canvas size (#956) 2019-10-01 12:25:21 -07:00
Bruce Martin 21218f1c59 update contact and core team metadata (#947)
* update contact and core team metadata

* fix spelling error
2019-10-01 10:26:38 -07:00
Bruce Martin 3d15db7c5e parameterize pbmc3k scanpy engine test (#939) 2019-09-20 17:41:32 -07:00
Bruce Martin 109c9e70ec Add obs index to label file (#928)
* remove debug logging

* load and save obs/row index in label file

* lint

* update tests
2019-09-19 09:16:01 -07:00
Bruce Martin 3660a6cc27 Experimental - manual annotations (#837)
* icons, partway

* redux for values

* onChange

* cancel

* annotations lifecycle for category names

* copy categorical

* edit category

* add Dataframe.withColsFrom

* render user annotations; default add/delete annotation category

* add label name to actions

* category name edit

* error checking improvements

* change schema field isUserAnnotation to writable

* always have an unassigned label; implement delete label

* implement add new label and edit label name

* label current cell selection

* fix select exact bug in crossfilter

* clean up categorical reducer

* fix tests

* remove debugging printf

* implement subset/reset for user annotations

* undo redo support for user annotations

* remove duplicate button from categories

* add modal

* remove obsolete duplicate annotation reducers

* remove old debugging printf

* connect modal to annotation create and dup

* initial full-stack wiring

* finish up end-to-end wiring

* fix existing unit tests

* fix pytests to match new schema API

* remove debugging printfs

* add label file rotation

* remove obsolete comment

* add fbs encode/decode tests

* add tests for writable annotations

* simplify code

* fix hashing bug with FBS encoding

* lint

* fix smoke tests

* improve error checking in Dataframe.withColsFrom

* add unit test for Dataframe.withColsFrom

* add unit test for Dataframe.columns and Dataframe.renameCol

* fix bug in FBS encode, add better error checks, refactor

* add FBS encode/decode test

* add clarifying comment

* clean up action type names; fix state inconsistency in crossfilter update

* change autosave timer to 2.5sec

* sort categorical metadata render order so it remains consistent

* add temporary autogenerated label for add-new-label operation

* fix hover-over label menu interference with cell highlighting

* remove debugging code

* add missing reducer cases & fix typo

* make dataframe memoize more general purpose

* add dev mode for annos

* fix error on select duplicate

* handle zero occupancy categories

* correctly maintain unclipped AND clipped world

* correctly handle zero length FBS matrix and label files

* ensure all writable categorical schema contains an unassigned category

* handle case where building occupancy stack for category with no members

* dialog for creating label, disable button if duplicate or empty

* visually separate writeable

* edit category

* fix edit category name

* remove debugging code

* fix edit annotation label

* visually define unassigned, change options

* Pull in requirements.txt from `master`

* label currently selected cells

* duplicate label

* lint

* fix pytest merge issues

* rename --label-file to --experimental-label-file

* remove debugging console log

* spelling error fix; fix bug found in PR review.

* lint
2019-09-18 07:33:41 -04:00
Bruce Martin ab2c423006 JS package updates (#922) 2019-09-17 09:36:47 -07:00
Bruce Martin 922c375333 Rename "layout" to "embedding" in the UI (#921)
* rename "layout" to "embedding" in the UI

* docs changes for layout

* fix typo in data docs

* new help text, per PR review
2019-09-16 15:34:14 -07:00
Bruce Martin 20317fd08f Add URL data locators to launch sub-command (#920)
* initial commit of URL support for launch

* lint

* modify tests to use new data locator

* add locator unit tests

* fix typo in faq

* more lint

* update faq per PR review
2019-09-15 09:01:53 -07:00
Bruce Martin 84c815cc04 pin h5py to 2.9.0 to temporarily work around regression (#916) 2019-09-11 13:20:15 -07:00
Bruce Martin c35f94fe6d add constraints on pan and zoom-over point (#904) 2019-08-28 17:56:46 -07:00
Bruce Martin 78a43402cb package dep update (#905) 2019-08-28 09:11:10 -07:00
Bruce Martin b0562a11fd scale point size for graph and scatter plots (#903)
* scale point size for graph and scatter plots

* improve comments
2019-08-27 14:12:53 -07:00
Bruce Martin b309367bb0 Add hover-over highlighting to the scatterplot (#900)
* small perf improvement - do not recreate functions each time component is updated

* add hover-over highlighting to scatterplot
2019-08-27 14:01:30 -07:00
Bruce Martin b359a610da tooltips improvements (#896)
* add and fix tooltips on toolbar

* move tooltips below buttons

* add global definition for tooltip hover over delay
2019-08-22 20:56:33 -07:00
Bruce Martin 0f36b49840 Graph/scatterplot improvements (#893)
* many graph/scatterplot fixes: 722, 882, 758, 886

* fix e2e test breakage caused by graph transform work

* convert webgl point shaders to use flags rather than explicit point size

* remove unused packages from dependencies

* fix typo in regl prop name

* factor common code into util helper file

* legibility tweaks
2019-08-22 20:56:05 -07:00
Bruce Martin 906d65c06f fix binning error in continuous data histogram (#869) 2019-08-05 10:40:56 -07:00
Bruce Martin e199b14259 correctly handle float columns which only contain NaN (#848) 2019-07-15 14:49:33 -07:00
Bruce Martin ca9a6796d8 release 0.10.1 (#797) 2019-05-30 13:44:42 -07:00
Bruce Martin a6142bdf93 improve graph scale and centering (#796)
* add gutter to embedding canvas

* improve layout scale and translate

* fix lint

* pin tables to version 3.5.1

* fix lasso coordinate smoke tests
2019-05-30 13:31:41 -07:00
Bruce Martin c6252825f3 release 0.10.0 (#794) 2019-05-29 16:55:43 -07:00
Bruce Martin 4b96b3a635 fix incompatibility of flask reload and port searching (#793)
* WIP

* add --developer; fix incompatibility of --port and --debug

* put REST tests on separate ports

* PR review
2019-05-29 16:38:57 -07:00
Bruce Martin 1ef77d1596 fix misconfiguration for history management (#787) 2019-05-24 21:01:11 -07:00
Bruce Martin 3dc45d6330 do not hard-wire column names in annotations (#785)
* enforce column name uniqueness for obs and var

* parameterize the column name containing obs and var user-readable names

* use the new annotation index value from schema

* update f/e unit tests

* PR review suggestions

* lint
2019-05-24 21:00:54 -07:00
Bruce Martin a8c2e408d1 update to latest anndata and remove restriction on scipy (#790) 2019-05-24 11:23:35 -07:00
Bruce Martin e941c1a496 scaling omitted from event handlers (#789)
* scaling omitted from event handlers

* fix smoke tests
2019-05-24 07:00:04 -07:00
Bruce Martin ef7c26e799 correctly handle selection of trunctated categories (#781) 2019-05-23 08:46:22 -07:00
Bruce Martin 49af278de7 cleanup memoiziation in graph component (#783) 2019-05-22 17:37:18 -07:00
Bruce Martin 2357d0c1b8 layout change UI (#776)
* add layout to schema

* add layout choice action and reducer

* multi layout UI

* update unit tests

* add missing file

* update test schema

* fix duplicate test id

* fix tabs

* PR lint

* fix pytest
2019-05-22 13:21:33 -07:00
Bruce Martin fcc05f6a00 coordinate system fixes for embedded graph (#768)
* change pan speed to 1 per issue #722

* correct handle scaling of graph when aspect ratio less than one

* add package lock

* add invert to our scale functions

* correctly transform to/from gl coordinates

* remove unused import

* fix naming of import

* update smoke tests
2019-05-20 14:22:41 -07:00
Bruce Martin de3407d875 change scripts to support windows (#775) 2019-05-20 11:42:57 -07:00
Bruce Martin 1fa4838863 npm (js) package dependency updates (#765)
* JS package dependency updates

* additional package updates

* more package version updates

* more js package updates

* more JS dependency updates
2019-05-20 10:11:49 -07:00
Bruce Martin efa1709158 add multi-layout support to back-end (#766)
* add multi-layout support to back-end

* remove obsolete code

* temporary code to apply heuristic choice of default layout

* fix tests

* update python tests

* more py lint

* PR review changes

* more PR lint

* PR lint
2019-05-16 14:49:22 -07:00
Bruce Martin b9a1e30652 large file size guardrails (#763)
* large file guardrails

* fix lint

* PR review

* remove unused import

* use standard slice for CSR

* revert change
2019-05-13 18:13:16 -07:00
Bruce Martin 7adac5d004 create occupancy stacks for all category values, not just top N values (#764) 2019-05-13 11:22:46 -07:00
Bruce Martin 846b8d15bd lodash cleanup (#747)
* add own range() function

* lodash cleanup

* remove redundant fill range implementations

* remove use of _.get

* sync test babel config with build

* update tests to match new range implementation
2019-05-06 20:28:32 -04:00
Bruce Martin 1471d6b214 release 0.9.0 (#746) 2019-05-04 08:41:56 -07:00
Bruce Martin 0aa0f641ab improve selection interaction with clip changes (#744)
* brush interactions with underlying dataframe updates improved

* improve comment

* reset selection state upon clip
2019-05-03 11:14:30 -07:00
Bruce Martin b521a17ffd improve column access speed for sparse matrices (#742)
* improve column access speed for sparse matrices

* add FAQ entry about data format performance

* add note about using --sparse flag for prepare command

* clean up for PR review

* Update docs/faq.md

Co-Authored-By: bkmartinjr <bruce@chanzuckerberg.com>

* improvements to big data faq
2019-05-01 15:44:01 -07:00
Bruce Martin 9044b8d85d fix color-by regression in toggle (#705)
* fix color-by regression in toggle

* fix incorrect field reference
2019-04-11 12:47:38 -07:00
Bruce Martin 5e02408732 latent bug in color toggle (#701) 2019-04-09 10:50:35 -07:00
Bruce Martin 7275d9d4dc Graph selection state management and history bug fixes (#679)
* save graph selection in redux state

* fix old graph brush select regressions

* refactor graph brush selection to work with undo/redo

* update tests to match new crossfilter spatial select API

* graph selection state now in redux

* remove dead code

* sync graph selection with redux state; improvements to undoable machinery

* fix regression in undoable

* differentiate graph selection cancel from deselect action

* simplify calculation

* remove debugging code

* fix responsive repaint bug in graph selection tool

* undoable debugging and code cleanliness

* undoable action filter state now merges, rather than replaces

* improve comments

* add debounce to undoable action filter; improve comments and debug sanity check code

* comments

* fix undoable bug with clear scatterplot actions

* disable undoable debug flag

* cleanup API and comments around statemachine

* add test id attribute to lasso

* add better error handling for gene fetch requests
2019-04-08 15:53:12 -07:00
Bruce Martin 71505abe4c toggle color-by when repeatedly picked by the user (#696)
* toggle color-by when repeatedly picked

* remove debugging code
2019-04-08 10:57:47 -07:00
Bruce Martin 153f240c43 brushable histogram brush selection responds to state updates (#670)
* add continuousSelection reducer and make histo respond to state changes

* correctly handle uninitialized state for brush move

* PR review tweaks
2019-03-25 15:02:39 -07:00
Bruce Martin 7ef5203564 Undo/redo (#659)
* immutable crossfilter

* first cut at reducer refactor with cascade model

* add initial redo/undo implementation

* small optimization

* integrate expression with history

* add tests for new reducers and fix a couple of small initialization bugs

* treat tiny lasso selections as a clear

* better function name for clarity

* fix undo for differential expression

* remove logging

* fix regression due to bad merge

* cleanup and comments for clarity

* improve undoable configuration for flexibility

* fix stale comments

* remove debugging code from production build

* rename categoricalSelectionState

* rename file

* improve comments
2019-03-22 14:53:48 -07:00
Bruce Martin 9420abfacc add Babel support for JS optioning chaining and nullish coallescing operators (#664) 2019-03-22 14:51:51 -07:00
Bruce Martin 996b06cecc refactoring - immutable crossfilter (#647)
* immutable crossfilter

* PR review changes
2019-03-20 14:32:35 -07:00
Bruce Martin 571b7387e7 remove dead code from globals (#651) 2019-03-19 06:22:39 -07:00
Bruce Martin aa5ce4a2f1 correctly toggle group selection in categorical metadata (#640) 2019-03-12 10:49:29 -07:00