1584 Commits

Author SHA1 Message Date
Isaiah Norton
9be086b1e8 Use sparse=True for from_pandas (#1994) 2020-12-02 13:39:19 -08:00
maniarathi
9a34264f92 Pin tiledb version to exclude 0.7.2 (#1993) 2020-11-30 11:54:33 -05:00
maniarathi
66e55ba59a Replace outdated locustio package with locust package. (#1992) 2020-11-30 10:11:57 -05:00
maniarathi
f700fb7757 Setup codeql scanning for security and code quality analysis (#1991) 2020-11-24 21:53:10 -05:00
maniarathi
ae310097dd Fix typo (#1990) 2020-11-24 12:23:21 -05:00
maniarathi
ea70a35a01 Fixing locust scale tests for cellxgene loading apis and adding a Github Actions workflow to run the tests every Sunday. (#1988) 2020-11-24 09:08:57 -08:00
maniarathi
16718f392f Sorry! Undo-ing commit to main 2020-11-24 10:22:24 -05:00
maniarathi
ad6b1928e4 Upgrade anndata and h5py 2020-11-24 10:21:24 -05:00
bmccandless
2cc02a84cb Convert float annotations if possible. (#1987)
* Convert float annotations if possible.

The client converts all arrays to floats.
If a category contains integer labels, and that category is copied, it will contains floats (e.g 1.0 instead of 1).
When that category is put back to the server, it fails in the tiledb code, which does not accept floats.
The solution is to convert a float category to integer, if possible.

  #1984

* updates
2020-11-20 17:01:53 -06:00
Marcus Kinsella
f77038ad58 Permit other keys in the cxguser cookie (#1982) 2020-11-18 17:58:33 -08:00
Timmy Huang
2cf55ab819 thuang-compress-annotation (#1980)
* thuang-compress-annotation

* compress test

* use zlib.decompress directly
2020-11-17 14:05:07 -08:00
bmccandless
095db02439 Remove deprecated health endpoint (#1943)
NOTE:  do not push to main until the new path has been terraformed into all the environments.

 #1846
2020-11-13 10:49:11 -08:00
dependabot[bot]
85fc000418 Bump dot-prop from 4.2.0 to 4.2.1 in /client (#1954)
Bumps [dot-prop](https://github.com/sindresorhus/dot-prop) from 4.2.0 to 4.2.1.
- [Release notes](https://github.com/sindresorhus/dot-prop/releases)
- [Commits](https://github.com/sindresorhus/dot-prop/compare/v4.2.0...v4.2.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com>
2020-11-09 22:05:34 -08:00
Isaiah Norton
34742b9a94 Add link to dev_docs in Contributing section (#1957) 2020-11-09 21:57:00 -08:00
Bruce Martin
06b88cda20 do not update GPU buffers if data has not changed (#1967)
Co-authored-by: maniarathi <mani.arathi@gmail.com>
Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com>
2020-11-09 10:41:39 -08:00
bmccandless
23714bc9f8 Fix bug that occurs when all categories are removed. (#1974)
Previously if the user remove all annotations, the code would still generate a tiledb uri
in the write_labels call, and add that to the database.  A tiledb array would not be written in this case.
When the read_labels was then called, it would find the entry in the database, attempt to open
the tiledb array, then fail.

The patch here will set the tiledb_uri to the empty string if all categories are removed.
When read_labels is called, it will see the empty uri and return None.
Furthermore, if the database does have a tiledb_uri that does not exist, or cannot be read,
then the code will now log a warning, and return None (instead of throwing an exception,
which results in a server error).

 #1932
2020-11-06 18:04:14 -08:00
Marcus Kinsella
e892e64685 Convert HGNC ids to their symbol (#1972)
There are entries in some var indexes like HGNC:18790. We'd like to convert that to its symbol, NSG1.
2020-11-06 09:27:49 -08:00
Marcus Kinsella
39a1124c35 Fix manifest to include schema definitions (#1965)
🤦

Co-authored-by: maniarathi <mani.arathi@gmail.com>
2020-11-05 09:41:05 -08:00
Bruce Martin
b5856ff9bc fix accidental state error (#1966) 2020-11-04 18:46:04 -08:00
Severiano Badajoz
d87551ce5e fix version check (#1968)
Inverted the schema version check so that we check that it IS 1.0.0.

![image](https://user-images.githubusercontent.com/8716829/98184898-5c897b00-1ec0-11eb-89f7-c293dfbe8439.png)
2020-11-04 17:43:41 -08:00
Severiano Badajoz
a5c9ae2432 Data drawer final round of prod QA fixes (#1955)
* add long title

* add organism to Dataset Metadata and create headers

* begin HTMLTable for metadata

* switch out truncating for scrolling

* add optional chaining to redux state mapping

Co-authored-by: maniarathi <mani.arathi@gmail.com>
2020-11-04 09:43:43 -08:00
Marcus Kinsella
78176f9711 Add schema subcommand (#1939)
Add the `cellxgene schema apply` and `cellxgene schema validate` subcommands.

The first takes an h5ad file and a yaml with config information and produces a new h5ad that follows the cellxgene data integration schema.

The second takes an h5ad and checks if it follows the schema version written into its metadata.

Both are currently marked as "experimental" as the primary intended users are still at CZI.
2020-11-02 08:26:37 -08:00
bmccandless
b9e132a00c Updates due dependency version changes. (#1960)
* Updates due dependency version changes.

h5py recently changes and now values once returned as str are now returned as bytes.
This would have caused a much larger change, so instead the version is restricted to <3.0.0.

This caused the bulk of the testing failues.
A few other changes were needed to make a few other tests pass.

 #1959
2020-11-01 12:36:38 -08:00
Madison Dunitz
3b6c46ba86 Fix dependency issues in compatibility tests (#1951)
* update reqs

* pin scanpy

* merge in fix for race conditions
2020-10-30 10:47:12 -05:00
bmccandless
6a1e5f71be fix race condition in test_oauth (#1956) 2020-10-29 11:05:23 -07:00
Severiano Badajoz
727af83152 remove conditional rendering cases from color legend (#1952)
* Revert "Remove Continuous vars with 1 value from histogram, add to info drawer (#1927)"

This reverts commit 242546371b.

* remove conditional rendering cases

* ignore pointer events

Co-authored-by: Madison Dunitz <madison.dunitz@chanzuckerberg.com>
2020-10-28 15:35:28 -07:00
Severiano Badajoz
924b518492 Revert "Remove Continuous vars with 1 value from histogram, add to info drawer (#1927)" (#1953)
This reverts commit 242546371b.
2020-10-27 17:12:13 -07:00
Madison Dunitz
946a910ef4 Fix compatibility test (#1948)
* update anndata version and warning about version

* update compatibility tests
2020-10-26 17:23:55 -05:00
bmccandless
7e9353c5f1 Fix bug in oauth. (#1949)
* Fix bug in oauth.

The error checking was too specific, and missed a case.
Make the error checking catch all exceptions.

  #1947

* Add logging when the cookie cannot be processed
2020-10-26 09:39:06 -07:00
bmccandless
c106ebc525 smnall fix to the test suite. (#1944)
I noticed a few tests failed when run individually, but not as a suite.

 #1942
2020-10-23 15:14:31 -07:00
Severiano Badajoz
2fa206f2ad Add token invalidation tests to oauth tests (#1941)
* add tests

* run black

* run black and add disclaimer that tweaked errors on server

* lint

* change to get so it will return None

* tweak existing token instead of new one

* Trigger

* token is dict

* jsonify dict before encoding

* json dump instead of jsonify

* encode into bytes object

* use correct id token

* decode byte to string
2020-10-23 14:51:48 -07:00
bmccandless
f41a023418 Minor changes to eb server to use Docker (#1938)
part of #1866
2020-10-22 17:04:08 -07:00
maniarathi
9793398737 Add in missing previous crossfilter which was causing the re-embedding feature to fail. (#1936) 2020-10-21 09:16:13 -07:00
maniarathi
377e4bccaa Remove errornous checking for converting float64 to float32. In reality the slight difference by downcasting is totally fine. (#1935) 2020-10-19 10:31:36 -07:00
bmccandless
6a741956e1 Update readme for eb server. (#1928)
* Update readme for eb server.

Update the README with new way of handling secrets.
Update portions that were out of date.
Add a section for Authentication and a placeholder for User Annotations.

Also remove an obsolete function that processes the AWS secrets.

  #1522

Co-authored-by: Madison Dunitz <madison.dunitz@chanzuckerberg.com>
2020-10-16 14:02:05 -07:00
Severiano Badajoz
c9f9549118 Adopt JS standards once userinfo data is in frontend (#1930) 2020-10-16 11:59:28 -07:00
Madison Dunitz
242546371b Remove Continuous vars with 1 value from histogram, add to info drawer (#1927)
* remove single val continous metadata from histogram, add to info drawer

* refactor to save singleContinuous values in state

* fix edge case, single continuous values reappeard in rsb when clipped
2020-10-14 12:46:24 -05:00
maniarathi
798976e4c1 Fix custom color handling (#1929) 2020-10-13 15:47:56 -07:00
maniarathi
5325495123 Speed up dataset drawer rendering (#1926) 2020-10-12 11:20:58 -07:00
Severiano Badajoz
beb46bf3df add and check system arg to state auth type in e2e test(#1924)
* add and check system arg to state auth type

* add tolower

Co-authored-by: czimergebot <35308261+czimergebot@users.noreply.github.com>
2020-10-09 12:42:02 -06:00
maniarathi
86ff48ae36 Revert "Allow columns encoded in float64 to be rendered as part of continuous value histograms. (#1905)" (#1925)
This reverts commit b048fd8d9a.
2020-10-09 10:09:32 -07:00
Timmy Huang
c01a2c72b6 thuang-1840-authn-prompt (#1911) 2020-10-08 16:57:53 -07:00
Severiano Badajoz
6677d0de56 disable profile picture (#1923) 2020-10-08 12:16:02 -06:00
Severiano Badajoz
c4c48b9a57 create e2e test for auth buttons (#1907)
This PR adds a few helpful additions regarding authentication.

Changes:
* e2e tests are now run on test_oauth via a passed config.yaml 
* node dev server correctly handles `/login` and `/logout` endpoints to make developing for auth easier
* Introduced auth e2e tests to check that buttons display and work
2020-10-08 10:02:40 -07:00
bmccandless
b5ec43c4b1 Add a function to check the configuration for errors. (#1919)
This can be used as a sanity check before a deployment:

  chanzuckerberg/single-cell#63
2020-10-08 08:44:09 -07:00
bmccandless
6c1756f852 Enhance the AppConfig with external config sources. (#1904)
* Enhance the AppConfig with external config sources.

The external config sources are currently environment variables
and AWS secrets manager.

The config file can be augmented with a section describing how
environmen variables and secrets can update config parameters.

benefits:
 - it will enable the config to draw from more than one secret.  This is useful
   for shared secrets between cellxgene and data portal, as well as auth0 secrets.
 - it will make it very straightforward to check the config before a deployment.

 Part of #1859
2020-10-07 15:38:42 -07:00
Severiano Badajoz
1c4c501c43 Auth UI tweaks (#1915)
* remove auth buttons and dataset info from info menu

* add auth buttons to menubar

* remove auth from top left

* new auth buttons

* move infomenu to lsb dir

* styling fixes

* feedback

* more feedback

Co-authored-by: Timmy Huang <thuang@chanzuckerberg.com>
2020-10-07 15:02:55 -07:00
bmccandless
cf77a8da9e Add "picture" to the /userinfo endpoint. (#1914)
* Add "picture" to the /userinfo endpoint.

This may be null or a URL.
 add picture for the test authentication method
2020-10-07 12:17:23 -07:00
Madison Dunitz
eb108feb37 Performance test annotations (#1908)
* make testing plan

* create annotaions sets for different num categories/dataset size

* annotation creation testing

* create scale and perf tests for annotations

* create make commands for tests

* get cell count if not set in test_datasets dict
2020-10-07 12:36:02 -05:00
Leslie
b386ca3425 Move link to cellxgene data portal higher (#1909) 2020-10-06 13:16:59 -07:00