Commit Graph

105 Commits

Author SHA1 Message Date
Charlotte Weaver
3d98797d8c Installer (#840) 2019-07-12 12:29:22 -07:00
Charlotte Weaver
5effe4bbbb removed --diffexp cli param (#826) 2019-06-26 14:46:58 -07:00
Charlotte Weaver
d5deb1579f [EASY] fix max-category-items (#813)
* fix max-category-items

* match default for max category items

fe had 1000, be had 100
2019-06-13 15:45:43 -07:00
Bruce Martin
a6142bdf93 improve graph scale and centering (#796)
* add gutter to embedding canvas

* improve layout scale and translate

* fix lint

* pin tables to version 3.5.1

* fix lasso coordinate smoke tests
2019-05-30 13:31:41 -07:00
Bruce Martin
3dc45d6330 do not hard-wire column names in annotations (#785)
* enforce column name uniqueness for obs and var

* parameterize the column name containing obs and var user-readable names

* use the new annotation index value from schema

* update f/e unit tests

* PR review suggestions

* lint
2019-05-24 21:00:54 -07:00
Bruce Martin
2357d0c1b8 layout change UI (#776)
* add layout to schema

* add layout choice action and reducer

* multi layout UI

* update unit tests

* add missing file

* update test schema

* fix duplicate test id

* fix tabs

* PR lint

* fix pytest
2019-05-22 13:21:33 -07:00
Bruce Martin
efa1709158 add multi-layout support to back-end (#766)
* add multi-layout support to back-end

* remove obsolete code

* temporary code to apply heuristic choice of default layout

* fix tests

* update python tests

* more py lint

* PR review changes

* more PR lint

* PR lint
2019-05-16 14:49:22 -07:00
Bruce Martin
b9a1e30652 large file size guardrails (#763)
* large file guardrails

* fix lint

* PR review

* remove unused import

* use standard slice for CSR

* revert change
2019-05-13 18:13:16 -07:00
Charlotte Weaver
8a94b1e086 fix #754 (#755) 2019-05-07 12:43:20 -07:00
Bruce Martin
b521a17ffd improve column access speed for sparse matrices (#742)
* improve column access speed for sparse matrices

* add FAQ entry about data format performance

* add note about using --sparse flag for prepare command

* clean up for PR review

* Update docs/faq.md

Co-Authored-By: bkmartinjr <bruce@chanzuckerberg.com>

* improvements to big data faq
2019-05-01 15:44:01 -07:00
Charlotte Weaver
9f10d8095a GUI app (experimental) (#730)
* add default config

* first pass

* flake8

* cleanup

* first pass at using qthreads

* cleanup

* WIP

* better error handling

* improved UI

* bugfix

* fix merge bugs

* import order

* cleanup

* make gui requirements optional

* pr review requested changes

* Update server/gui/main.py

Co-Authored-By: csweaver <charlottesweaver@gmail.com>

* pr review request

* qt child class methods -> camelCase

* whitespace
2019-04-30 12:52:15 -07:00
Charlotte Weaver
ea187f48e0 add scripts from cli (#680)
* add scripts from cli

* add warning when including scripts

* flake8 fixes

* confirm scripts injection
2019-04-22 12:30:57 -07:00
Charlotte Weaver
9c6273eb94 core library (#711)
* move app creation to function

* create engine without load

* flake 8 fixes

* cleanup original scanpy test

* add default config

* handle missing data

* test data changes

* unify update

* load data isn't static anymore

* make app a class
2019-04-22 12:24:07 -07:00
Charlotte Weaver
73852dd6f4 use flask's json (#703) 2019-04-09 12:59:09 -07:00
Sidney Bell
c9a56fa73a Update scanpy version (#688)
Update to version 1.3.7
2019-04-04 12:16:30 -07:00
Sidney Bell
16f93397ae Support diffmap and phate layouts. Explicitly handle embeddings with >2 components. (#662)
* Add diffmap and phate to supported embeddings

* Explicitly pull the first two components of any given layout
2019-03-21 19:06:05 -07:00
Charlotte Weaver
e875ed739b More informative out-of-memory error (#644)
* more informative error message: memory error

* flake fix
2019-03-12 14:19:00 -07:00
Colin Megill
a0f54b4871 Add menu (#631)
* add menu

* Added library versions to config

and tests

* add template version number
2019-03-06 11:56:46 -05:00
Bruce Martin
57c4e9ff33 Flatbuffer cleanup (#598)
* dead code and route removal

* more dead code cleanup

* fix scanpy_engine tests

* lint

* add missing catch in filter parsing

* update scanpy NaN tests

* more fbs tests and dead test removal

* remove forced default for content type negotiation

* bit of cleanup

* more fbs test cleanup

* lint

* remove swagger

* swagger cleanup

* lint

* correctly handle lack of templates

* more dead code removal

* remove unused files

* fix dev build

* lint
2019-02-19 08:50:29 -08:00
Isaac Virshup
0f8d7a55de Set API path based on access address (#568)
* Make api paths relative

* Remove request import

* Set publicPath to be relative
2019-02-11 09:01:37 -08:00
Charlotte Weaver
10693b08cc Add __init__ file so fbs can be imported (#562) 2019-01-17 17:11:31 -08:00
Bruce Martin
5d60505407 remove --nan-to-num CLI parameter (#548)
* remove --nan-to-num CLI parameter

* factor tests better

* lint - remove unused variables
2019-01-10 15:03:03 -08:00
Bruce Martin
b90447c387 binary wire format with flatbuffers (#509)
* first flatbuffer schema

* do not lint auto-generated files

* add flatbuffers package

* add flatbuffer module

* wire up /data/X/T route

* use flatbuffers for matrix data fetc

* clarity and comments

* add flatbuffer layout route

* clean up obsolete code

* fix tests

* move flake8 config to setup.cfg

* add comments

* lint

* rework layout routes for fbs

* add more type support to fbs

* lint

* add flatbuffer support for annotations

* function name improvements

* fix botched merge with master

* remove unused import

* route cleanup for flatbuffers

* rename function for clarity

* add missing globals to Jest tests

* fix client JS tests

* fix routes for Python tests

* comments for clarity

* non-finite floating point hardening

* more non-finite number handling

* lint

* fix tests for summarizeAnnotations

* harden diffexp calculation against FP errors

* cleanup unused code

* lint

* add encoding tests for flatbuffers

* application type specified as strings

* fix spelling error

* improve variable names

* add note about documentation gap

* rename FBS DataFrame to Matrix
2019-01-09 14:26:05 -08:00
James Taylor
f2eb2cad82 Use url_root from Flask rather than hardcoding hostname. (#520)
Allows running in a container on a remote host, should also allow running
behind a proxy with a url prefix.
2018-12-19 16:20:26 -08:00
Charlotte Weaver
f0f7200f0b Move jsonification to engine level (#511) 2018-12-18 21:14:24 -08:00
Charlotte Weaver
83154577e4 Black -- formatter for python (#508)
* Add black

* use black to format code

* Black version
2018-12-12 09:44:47 -08:00
Charlotte Weaver
5c6cc597c8 Default warning in debug mode (#497) 2018-12-05 13:38:34 -08:00
Bruce Martin
3bfeadc2b9 improve handling of non-finite floating point values (#490)
* improve HTTP error reporting

* generate standards-compatible JSON

* add --nan-to-num work-around for non-finite floating point values

* lint

* update tests

* correctly set Infinities to min/max

* REAMDE update for --nan-to-num

* define constant for repetitive warning message

* clarify where NaN errors will occure
2018-12-04 14:56:16 -08:00
Bruce Martin
af0d1f6fb2 issue #480 workaround (#484)
* only load annotation var names

* remove incorrect usage of var annotation data

* temporary workaround for issue #480

* lint

* issue warnings only once per item
2018-11-29 16:33:21 -08:00
Charlotte Weaver
76c0cbf16b loose -> lose (#479) 2018-11-28 13:07:10 -08:00
Charlotte Weaver
455d987398 Add custom format for warnings (#477)
[cellxgene] Warning: <message>
2018-11-28 12:47:11 -08:00
Bruce Martin
933fac5fff Allow float64 to down-cast to float32 (#472)
* warn if annotations will down-cast to float32

* correctly handle falsey data values
2018-11-27 11:17:30 -08:00
Bruce Martin
138d30909a improve type handling for non-string annotation data (#465)
* improve type handling for non-string annotation data

* improve clarity of code
2018-11-26 10:32:29 -08:00
Bruce Martin
141f802824 differential expression improvements (#452)
* add cutoff for low expression genes in topN selection

* remove debugging printfs

* change cli param name for diffexp cutoff

* change CLI param name

* second try at diffexp - using lfc sort with pval cutoff

* use lfc cutoff

* update comments to match code; cap p-value adjustment to max of 1

* lint

* explain diffexp in readme

* add link

* add diffexp-lfc-cutoff to test config

* update test to match revised diffexp spec

* fix latent bug in GET arg handling that was breaking tests

* lint

* comment cleanup

* fix variance overestimation so it is symmetric

* lint
2018-11-16 14:44:56 -08:00
Bruce Martin
00a68276a2 diffexp performance & UX improvements (#431)
* new diffexp REST API spec

* new diffexp REST API; faster diffexp and dataframe slicing

* first draft of fast diffexp

* convert variance calculation to two-pass method

* lint

* update front-end use of API

* fix typo in spec

* disable content compression

* catch index filter format errors

* clean up of dead code

* resolve PR review comments
2018-11-14 12:51:24 -08:00
Charlotte Weaver
7d5c054b90 Launch validation (#414)
* WIP

* Cleanup

* Validation

* typo

* Max category limit 100 -> 500
2018-11-07 13:01:13 -08:00
Bruce Martin
2d5bc9d0c9 Performance work, plus fix #405 (#406)
* remove memoization

* update to flash 1.0.2; turn on threading

* stand-alone helper routines for array slicing

* fix issue #405

* reset diffexp state when world changes

* performance work in dimension creation; fix world slicing bug

* update tests to match new state mgmt api

* update flask

* do not make dimensions for useless annotations

* update test to match optimizations
2018-11-05 13:57:58 -08:00
Jeremy Freeman
0c8a07ac13 CLI refactor (#396)
* refactor cli to improve ux and enable easy incorporation of prepare as a subcommand

* switches to use click, which removes some boilerplate and gets us some improved ux for free

* changes the entry point for the cli

* changes the name of the browser option to --open and makes the default false
2018-11-02 10:52:50 -04:00
Bruce Martin
b181751493 add --obs-names and --var-names CLI params (#371)
* add --obs-names and --var-names CLI params

* fix lint

* performance improvements in scanpy engine

* fix lint

* fix typo

* correctly handle sparse formats in diffexp

* fix diffexp and 1d slicing

* diffexp uses t-stat, not pval; clean up arg handling

* make _slice a static method

* revise scanpy tests to match new API
2018-10-30 14:07:38 -07:00
Charlotte Weaver
26d222cd75 Move computation options to app (#386)
no need to import scanpy engine for CLI help so speeds up time response
2018-10-26 17:07:21 -07:00
Charlotte Weaver
6c23a72e5f CLI renaming and phrasing (#385)
* Minor naming and phrasing changes from UX review

* category-selection-limit -> max-category-items
* Indicate load may taking a long time
* program -> command (for launch, prepare)
* debug -> verbose
* flask-debug -> debug

* Developer mode for debug

verbose on
open browser off

* move examples from epilogue to prefix
2018-10-26 15:59:34 -07:00
Charlotte Weaver
16b0741a6c Hide stack traces unless in debug mode (#384) 2018-10-26 13:36:53 -07:00
Charlotte Weaver
94f95d6565 CLI Launch (#366)
* Scanpy engine now required

Without the --engine param we need to error if scanpy engine cannot be imported rather than waiting for all engines

* CLI options and help matches proposal

(but not all options hooked up yet)

* Flesh out top level args

* Move computation args to engine

* CLI input file (#374)

* Fix test command

(tests still won't work)

* Input is file instead of directory
- also renamed example file

* Csweaver/debug (#376)

* Respect debug flag for logging flask calls

* Add loading messages

* max categories (#377)

* Add max categories

* Rename max_categories to category_selection_limit

* ensure whole numbers
2018-10-24 19:28:59 -07:00
Charlotte Weaver
85f44ccabf Converts NaNs in pval to 1s (#365) 2018-10-24 10:57:29 -07:00
Bruce Martin
3556bd11f5 fix lint (#369) 2018-10-24 07:12:51 -07:00
Charlotte Weaver
efb55a6332 Renaming bind-all option to listen-all (#353)
See discussion here https://docs.google.com/document/d/1tcAvODhdlUUAOJPHoGQB3fWOPmnw3XjK7MVhDRwNUnk/edit#
2018-10-19 15:26:19 -07:00
Charlotte Weaver
2a3577f32b flask debug as hidden option (#352)
* Remove flask debug flag

* Add debug flag as a hidden option to keep reload on change for developers
2018-10-19 10:24:15 -07:00
Charlotte Weaver
06f402ab0a Do not calculate layout, used saved layout instead (#343)
* Do not calculate layout, used saved layout instead

See for rationale: https://docs.google.com/document/d/1HJFvbdDHxxgkCW0DZzdTZ9CUMgc2ef2rQFukATBQWvE/edit

* Error handling for when layout has not been precomputed

* Server error (500) not client error (400) for unprepared data
2018-10-18 11:51:38 -07:00
Charlotte Weaver
7c8b89eba3 Launch browser to cellxgene (#325)
* Launch browser to cellxgene

Also added  --no-launch command line parameter

* Don't launch browser for tests

* rename no-launch to no-open

* Rejigger -no-open to positive destination

so later logic looks cleaner
2018-10-17 16:46:56 -07:00
Bruce Martin
76ec29734a Performance work (#334)
* range encode filter range lists

* speed up data load

* add comment on scanpy read params

* update to latest scanpy/anndata

* performance improvments in data loading

* fix typo

* work around scanpy bug

* remove debugging print statements
2018-10-16 15:49:40 -07:00