Previously if the user remove all annotations, the code would still generate a tiledb uri
in the write_labels call, and add that to the database. A tiledb array would not be written in this case.
When the read_labels was then called, it would find the entry in the database, attempt to open
the tiledb array, then fail.
The patch here will set the tiledb_uri to the empty string if all categories are removed.
When read_labels is called, it will see the empty uri and return None.
Furthermore, if the database does have a tiledb_uri that does not exist, or cannot be read,
then the code will now log a warning, and return None (instead of throwing an exception,
which results in a server error).
#1932
Add the `cellxgene schema apply` and `cellxgene schema validate` subcommands.
The first takes an h5ad file and a yaml with config information and produces a new h5ad that follows the cellxgene data integration schema.
The second takes an h5ad and checks if it follows the schema version written into its metadata.
Both are currently marked as "experimental" as the primary intended users are still at CZI.
* Updates due dependency version changes.
h5py recently changes and now values once returned as str are now returned as bytes.
This would have caused a much larger change, so instead the version is restricted to <3.0.0.
This caused the bulk of the testing failues.
A few other changes were needed to make a few other tests pass.
#1959
* Fix bug in oauth.
The error checking was too specific, and missed a case.
Make the error checking catch all exceptions.
#1947
* Add logging when the cookie cannot be processed
* add tests
* run black
* run black and add disclaimer that tweaked errors on server
* lint
* change to get so it will return None
* tweak existing token instead of new one
* Trigger
* token is dict
* jsonify dict before encoding
* json dump instead of jsonify
* encode into bytes object
* use correct id token
* decode byte to string
* Update readme for eb server.
Update the README with new way of handling secrets.
Update portions that were out of date.
Add a section for Authentication and a placeholder for User Annotations.
Also remove an obsolete function that processes the AWS secrets.
#1522
Co-authored-by: Madison Dunitz <madison.dunitz@chanzuckerberg.com>
* Enhance the AppConfig with external config sources.
The external config sources are currently environment variables
and AWS secrets manager.
The config file can be augmented with a section describing how
environmen variables and secrets can update config parameters.
benefits:
- it will enable the config to draw from more than one secret. This is useful
for shared secrets between cellxgene and data portal, as well as auth0 secrets.
- it will make it very straightforward to check the config before a deployment.
Part of #1859
* make testing plan
* create annotaions sets for different num categories/dataset size
* annotation creation testing
* create scale and perf tests for annotations
* create make commands for tests
* get cell count if not set in test_datasets dict
* Remove door icon from log in button
* Move log in and info buttons from the top bar to in line with the cellxgene icon and dataset name
* Hover over on login button should say "Log in to cellxgene"
* Show email
closes#1830
* split out config
* add tests for base and app config, refactor client config out of app config
* refactor default config retrieval
* create config test class and helper functions
* move default_config into server to fix import issue
The config file had a bug where it expected both a "server" and "dataset" section.
If one didn't exist, then it would raise an exception.
It should use the default server config or the defaul dataset config in those cases.
Added a test case that would have caught this.
* app config bug fix:
When reading a config file that included per_dataset_config,
the dataroot specializations were applied, but not the default config.
This PR fixes that and also includes a test for this case.
* Fixes from frontend/backend url separation
This fixes the CORS and CSP headers.
Also, in thie commit, I removed the cors_supports_credentials config parameter,
which was recently introduced.
Instead, the logic determines the need to use CORS headers if the
web_page_url is set.
#1778
* separate backend base url from frontend
This is needed for auth, and to support a different location for the backend api server,
than the frontend.
part of chanzuckerberg/cellxgene#1778
new server config parameters: app__api_base_url, app__web_base_url
Also changed api_base_url in the oauth config section to "oauth_api_base_url" to
be less confusing with the app's api_base_url
Other minor changes:
changed how the jwt decode options are handled.
Previously they needed to be set in a test case, and there was some extra logic to handle that.
Now they are handled through comfig parameters, which makes it more general.
Also, add a feature to set the CORS support credentials, which seems
to be necessary for the backend/frontend separation, at least when run
locally. This part is sort of experimental, and may be removed or changed later.
When generating a config file, you can do this:
> cellxgene launch --dump-default-config > myconfig.yaml
And then modify the myconfig.yaml.
However, if an upgrade is available then you would get extra lines in the yaml
file, which are not yaml code:
There's a new version of cellxgene available (0.16.4)!
To upgrade, run the following: pip install --upgrade cellxgene
To solve this problem, the upgrade messages are sent to stderr instead,
so they will appear on the screen and not in the config file.
Alternatives:
One workaround is "cellxgene --no-upgrade-check launch --dump-default-config > myconfig.yaml"
But that's a bit verbose and not user friendly.
The way we've setup the upgrade check to be separate and before the launch sub command,
makes other code changes more involved.
#1826
* Fix frontend mishandling of null userinfo
If the authentication is disabled, the userinfo endpoint returns null.
This case needs to be handled.
#1780
* Small fix for handling refesh tokens in auth
* save tiledb array to s3, dont cache user annotations
* Add option to disable annotation filename prompt (#1787)
Co-authored-by: Madison Dunitz <dunitzm@gmail.com>
* set tiledb default context in cxg_adaptor
Co-authored-by: maniarathi <arathi.mani@chanzuckerberg.com>
Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com>
* Handle the refresh token in oauth authentication
If the token has expired, then it can be refreshed to get a new token.
This is automatically handled by the server without the client being aware.
Also in the PR:
- refactor the auth_oauth.py file to more simply handle the save/restore of the token,
and the refresh token
- added an end2end test for oauth, which also tests refresh.
* adding python-jose and Authlib to requirements-dev.txt
They are needed in the auth_oauth test
* Separate userinfo from the config endpoint
previously information about if the user was logged in and their username
was part of the config endpoint.
However, the config endpoint was previously static, and has a cache control.
Rather than not caching the config, a new endpoint called "userinfo"
is created to handle that information.
The config endpoint still has the non-changing part of the authentication:
config:
authentication:
requires_client_login: True/False
login: <uri to login endoint if requires_client_login is True>
logout: <uri to logout endoint if requires_client_login is True>
The userinfo endpoint returns this information:
userinfo:
is_authenticated: True/False
username: <string if is_authenticated>
if authentication is not enabled then the config does not have an authentication key,
and userinfo returns None.
Also in the PR are a few minor code improvements and bug fixes
Co-authored-by: Colin Megill <colinmegill@gmail.com>
bmccandless