Commit Graph

1085 Commits

Author SHA1 Message Date
Bruce Martin
8fac40b6ae various fixes for s3fs use (#1312)
* various fixes for s3fs use

* lint
2020-03-28 22:26:03 -07:00
bmccandless
54a75ffd7e Add support to include static files into the eb deployment (#1305)
Mostly this is just instructions for how to do this,
with a small addition to the makefile.
This enables support for serving the about_legal_tos and about_legal_privacy
from the cellxgene server.
2020-03-26 15:03:37 -07:00
bmccandless
8c0480b0bc Added a config hook for secret key into the app. (#1304)
* Added a config hook for secret key into the app.

the server first looks in an environment variable,
then looks in a config file.

For the cellxgene launch app, a default key is used if none is provided.
For the eb app, a secret key must be provided.
2020-03-26 15:03:08 -07:00
Bruce Martin
a398f85ccc turn off auto-reload in debug mode (#1301) 2020-03-26 13:34:56 -07:00
Bruce Martin
57938b487b remove static route redirects / bump static to top level (#1299)
* add another favicon rule

* catch only top-level dataset names

* add config flag to disable redirects

* remove static redirect, point all links to top-level /static

* remove extraneous argument

* remove httpd redirect
2020-03-26 10:45:10 -07:00
bmccandless
40c2e4c164 Change route to the datasets (#1302)
Previously is was /<dataset>/...
Now there is an extra levle:  /d/<dataset>/...
2020-03-26 10:35:17 -07:00
Colin Megill
adf7010f50 Add TOS and Privacy url via config (#1300)
* add tos and privacy url via config

* typo

* readd condition

* param
2020-03-26 11:33:41 -04:00
bmccandless
91e17e64a5 Fixes for static_rewrite.config (#1298)
replace tabs with spaces (yaml parsing complained).

change to favicon.png.
2020-03-25 17:44:15 -07:00
Bruce Martin
26605049a4 Various hardening to REST routes (#1293)
* URL reweriting for static

* request size limits

* improve quotas, make tests work

* remove debugging code

* pass limits to front-end

* fix renaming boggle
2020-03-25 16:14:52 -07:00
bmccandless
7b53209ae3 Fix a deadlock when an exception is thrown during matrix validate/open (#1296) 2020-03-25 15:49:49 -07:00
Bruce Martin
674f14f9bd harden dataset name handling (#1297) 2020-03-25 15:37:06 -07:00
Matt Weiden
e07ebf85e8 Small documentation fix in prepare (#1292)
See sample of current behavior below:
```
venv❯ cellxgene prepare example-dataset/pbmc3k.h5ad
[cellxgene] Starting CLI...
Error: [cellxgene] cellxgene prepare has not been installed. Please run
`pip install cellxgene[prepare]` to install the necessary requirements.

~/workspace/cellxgene mweiden/446-custom-color-palette*
1 venv❯ pip install cellxgene[prepare]
zsh: no matches found: cellxgene[prepare]
```

Fix:
Wrap cellxgene[prepare] in single quotes.
2020-03-25 14:38:07 -07:00
bmccandless
5ec66c5b0e Avoid race condition in the matrix cache handling. (#1280)
* Improved fix for matrix cache handling.

During the MatrixDataCacheItem acquire function there was a
time when the write lock was released and the read lock was taken.
During that time, the dataset could have been deleted, later
result in the MatrixDataCacheManageri data adaptor returning None.

The solution is to demote the writer lock to a reader lock instead
of unlocking and relocking.

Also, when a the cache needs to delete an entry, the delete
is done outside the MatrixDataCacheManager lock.   This operation
only requires the write lock for the MatrixDataCacheItem.

Fixes #1255
2020-03-24 12:23:47 -07:00
bmccandless
bea1836386 hosted cellxgene, add an environment variable to specify the config file (#1288)
Fixes #1272
2020-03-24 11:43:35 -07:00
Bruce Martin
752b9e4ab3 CORS and CSP headers (#1286)
* do in-app compression only for CLI

* CORS and CSP headers

* lint

* add --debug to targets

* lint

* fix botched merge with master
2020-03-24 08:05:37 -07:00
Bruce Martin
d196cbc8f6 do in-app compression only for CLI (#1285) 2020-03-24 06:36:40 -07:00
Bruce Martin
96b092f0eb fix health root exception (#1283)
* fix health root exception

* remove extraneous lambda
2020-03-23 15:13:32 -07:00
bmccandless
785bf5b5bc update for the legal configuration options (#1281)
* Configuration for legal information
2020-03-23 13:25:17 -07:00
Severiano Badajoz
b249df9036 add/fix label tooltips (#1278)
* Move tooltip to right

* Change function to only return truncated string

* add tooltip to category label

* use position instead of PopoutPosition
2020-03-23 10:28:54 -07:00
Bruce Martin
d99b84ba09 prepare - work around anndata bug (#1260)
* work around anndata bug 344

* fix accidental cut and paste error

* Use modified make_index_unique function

Temporarily copy code from https://github.com/theislab/anndata/pull/345
until the issue is resolved and released.

* Add notes and test for make_index_unique

* Lint fix

* Format python

Co-authored-by: Matt Weiden <538456+mweiden@users.noreply.github.com>
2020-03-22 12:27:59 -07:00
Bruce Martin
db7a485796 tighten up error reporting (#1269)
* black reformat

* tighten up error reporting

* lint

* fine tuning

* additional improvements in exception handling

* lint

* include exception and traceback in log

* fix typo
2020-03-22 09:55:47 -07:00
bmccandless
8180be83b8 Introduce a config file to cellxgene (#1264)
* Introduce a config file to cellxgene

The config file format is in yaml.  The default config is located
in server/common/default_config.py.  A user may create a yaml file
that contains a subset of these fields.  It can be used during cellxgene
launch, or for hosted cellxgene.

The code has also been refactored.  Much of the logic to check arguments
has moved from launch to app config.

It is now possible to set the tiledb context parameters using the config
file.  Other feature will soon be handled in a similar way.
2020-03-22 09:34:11 -07:00
Bruce Martin
1351c8f724 restrict methods on all endpoints (#1271)
* restrict methods on all endpoints

* lint
2020-03-22 08:23:28 -07:00
Colin Megill
c4731d52e2 target blank (#1254) 2020-03-20 20:07:51 -04:00
Bruce Martin
de006f8f91 initial implementation of a health check route (#1262)
* initial implementation of a health check route

* remove debugging code

* lint
2020-03-20 14:21:32 -07:00
Bruce Martin
040dbd3a95 refine load test (#1257) 2020-03-20 14:09:12 -07:00
Bruce Martin
8f09a4b1bc JS package fixes (#1248)
* Remove accidental package install from build

* remove accidental package install
2020-03-18 19:00:28 -07:00
bmccandless
e4bf65c54a Improve hosted cellxgene (#1234)
* Improve hosted cellxgene

 - option to turn off the test index page, or supply a page for redirect.
   For EB, The default is to return 404.  For cli launch, the default is the test page.

 - option to select which matrix types are allowed for multi dataset servers.
   For EB, The default is CXG only.  For cli launch, the default is any matrix type.

 - Return early with an error response if diffexp is requested when not configured

 - Verified that reembedings and user annotations also return with an error response
   if used when not enabled.

TODO:  The new options cannot currently be set by the user.
I plan to add a configuration file where these and all other settings can be set.

 Fixes #1210 
 Fixes #1228  
 Fixes #1229
2020-03-18 16:21:03 -07:00
Bruce Martin
38c4ecac8c locust load test script (#1244)
* initial version of a locustfile driver

* README
2020-03-18 11:27:07 -07:00
bmccandless
8a733270d7 fixes for hosted cellxgene (#1227)
* Fix for favicon with --dataroot

* fix static assets in hosted cxg 

The web proxy at aws eb was not finding the static assets.

The solution here is very simple: just copy the directory
containing the static assets to the top level of the artifact.zip.

This is not really the ideal solution.  According to the AWS
docs you can make a mapping to the correct location in an
an ebextentions config file.  I tried this and many combinations but
was not able to get this to work following that pattern.

Since we control the construction of the zip file, the solution
here isn't bad, but it could probably be made better.
2020-03-16 09:47:01 -07:00
Matt Weiden
86cbe64ae8 Add color mapping to the bar chart (#1218)
* Add color mapping to the bar chart.

* Change histogram color generation from Viridis to InterpolateCool.

* Coloring of Histrogram based up the x axis instead of y axis.

Coloring of Histrogram based up the x axis instead of y axis.

* Respond to feedback from @colinmegill

* Only color histograms that are selected for colorby
* Add some small refactors to BrushableHistogram's componentDidUpdate

* Fix histogram coloring and binning

* Reuse binning functions from util/dataframe/histogram.js; this fixes
an issue with there being near-zero width bins
* Fix color mapping so that it matches the scale in the legend
* Do not attempt to plot bins if the calculated binWidth is zero; this
can happen if all values are the same
* Refactor the function that draws the histogram a bit

* Respond to feedback from @bkmartinjr

Co-authored-by: Donald Paul Herman <Donaldpherman@hotmail.com>
2020-03-14 15:24:52 -07:00
Bruce Martin
1e8d89f8f2 refinements for 0.15 (#1226) 2020-03-13 16:30:47 -07:00
Severiano Badajoz
9c2f57df39 bump version to 0.15.0 (#1222) 0.15.0 2020-03-13 13:11:12 -07:00
Bruce Martin
9bc83fdd02 update contributor process recommendations (#1213)
* update contrib process recommendations

* fix typos

* clarification

* fix build instructions

* updates suggested by Matt

* fix dead link
2020-03-13 11:41:30 -07:00
bmccandless
bbfd2e5ac5 Add a simple annotations cache to the server (#1220)
Reading the user annotations had a performance problem.
The same file was read over and over.  Now, the last set of user
annotations are cached.
2020-03-12 20:06:44 -07:00
Bruce Martin
f28bd52d79 fix performance regression caused by excessive rendering (#1219) 2020-03-12 15:05:08 -07:00
Bruce Martin
63f1f38f7c use AnchorButton so the tooltip dismisses (#1217) 2020-03-12 08:36:36 -07:00
Severiano Badajoz
3ba4cf4483 fix bug created from master rebase (#1216) 2020-03-11 15:20:20 -07:00
Matt Weiden
a862f2b2a4 Further refine compatibility tests (#1215)
* Make sure apt is up to date before pulling hdf5

* Only install py dev reqs in cxg release vs anndata master test

* Don't need bu flag when using sed on ubuntu

* Don't re-install package reqs in python x anndata ver tests

* Minor documentation fix
2020-03-11 14:16:05 -07:00
Severiano Badajoz
3bf72829c2 pass empty coordinates when a continuous annotation is colored (#1212) 2020-03-11 13:01:14 -07:00
Severiano Badajoz
7cd9a0032a optimize centroid util (#1147)
* move unvarying evaluations outside of loop

* refactoring

* remove perf checks

* commenting

* minor fix + renaming

* small fix

* benchmarking

* merge master

* Revert pref checks

* renaming and comment

* remove redundant sets in Map
2020-03-11 12:35:42 -07:00
Sidney Bell
9089fc98f2 annotations cli updates (#1190)
* Switch logic from `--annotations` to `--disable-annotations

* Rename `--annotations-file` --> `--annotations-input-file` and remove `experimental`

* update docs

* update makefile

* update tests

* Disable annotations on standard client smoke test

* Update docs/posts/annotations.md

Co-Authored-By: Matt Weiden <538456+mweiden@users.noreply.github.com>

* Update docs/posts/annotations.md

* Renaming

* Docs

* Update tests

* Pesky typo -_-

Co-authored-by: Matt Weiden <538456+mweiden@users.noreply.github.com>
2020-03-11 11:57:52 -07:00
Bruce Martin
8beeb57c96 Use DataLocator consistently on backend (#1208)
* allow DataLocator to accept another locator as init param

* migrate to DataLocator

* migrate to DataLocator

* lint

* migrate to DataLocator

* add check for erroroneous use of remote path and annotations

* lint

* revert default data location - now back go CWD

* remove unused import
2020-03-10 19:12:49 -07:00
Colin Megill
234f25b782 Conditionally truncate category string (#1206)
* maybe truncate string

* add string formatting to test

* correct import

* destructuring

* add maxlength

* test

* Respond to feedback from @bkmartinjr

Co-authored-by: Matt Weiden <538456+mweiden@users.noreply.github.com>
2020-03-10 18:14:26 -07:00
Bruce Martin
28ddb60b2f update JS deps (#1203) 2020-03-10 10:59:12 -07:00
Colin Megill
1a93068810 Prevent category collapsing on annotation edit, # of cells on add label button (#1204)
* collapse functionality

* expand collapse
2020-03-10 13:21:06 -04:00
Matt Weiden
d8231a9a3b Fixes to compatibility tests (#1205)
* Unpin anndata ver in tests against anndata master

* Add branch config to work on push

* Add links to tests in GitHub Actions status badges

* Remove temporary branch name
2020-03-10 08:19:36 -07:00
Bruce Martin
ffdcc24a40 remove pinned tables dependency (#1202) 2020-03-10 07:47:54 -07:00
Severiano Badajoz
023a73cab8 smoke-test centroids + graph overlay layer (#1189)
* add test labels

* prettier + add clcik return to clickOn()

* prettier + begin test

* finish label counting test

* add util to get coordinates of element

* add test id to labels

* add test to check overlay transform

* remove logs

* rename to match master
2020-03-09 17:01:22 -07:00
Bruce Martin
144b19c449 experimental re-embedding (#1186)
* first cut at re-embedding route and back-end support

* update and expand config route tests

* add scanpy_umap

* add reembedding to config route parameters

* front-end support for reembedding fetch and UI

* remove unused imports

* add loading state

* save reembedding in reducer state

* improve withColsFrom

* transmit reembed schema to client; pick unique embedding names

* display embeddings

* format

* lint

* spaces, tab size 2

* lint

* test hack for smoke-test race

* back out hack sleep

* add check for backed mode

* add unit test for reembedding

* lint

* hide re-embedding CLI param from help
2020-03-09 16:53:30 -07:00