fork/cellxgene
1
0
Fork 0
mirror of https://github.com/chanzuckerberg/cellxgene.git synced 2026-09-29 08:28:11 +08:00
Code Issues Packages Projects Releases Wiki Activity

hosted, update order to look for config file. (#1452)

Browse Source 
* hosted, update order to look for config file.

The app now uses a local config.yaml file bundled with the artifact
(if present), if it exists, then looks in the CXG_CONFIG_FILE
environment variable.  This is the reverse of previous behavior.
The purpose of this change is to move away from using the
config file on s3, since that could lead to problem where an older
version of the app uses a newer version of the config.

Also in this PR:
1. Changed documentation around dataroot, to describe the posibility of using lustre.
2. Added a few improvements around the secret manager region name.  If we use lustre for dataroot and a local config file, then we will no longer be able to
auto determine the region for the secret manager.  I plan to start using the
environment variable option for hosted cellxgene.

* small edit to README

Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com>
This commit is contained in:
bmccandless
2020-05-05 10:50:28 -07:00
committed by GitHub
co-authored by Severiano Badajoz
parent 1ee76826c7
commit 61ca75e846
3 changed files with 69 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
server/eb/app.py
+37 -25
View File
@@ -124,43 +124,55 @@ class WSGIServer(Server):
try:
app_config = AppConfig()
dataroot = os.getenv("CXG_DATAROOT")
config_file = os.getenv("CXG_CONFIG_FILE")
has_config = False
# config file: look first for "config.yaml" in the current working directory
config_file = "config.yaml"
config_location = DataLocator(config_file)
if config_location.exists():
with config_location.local_handle() as lh:
logging.info(f"Configuration from {config_file}")
app_config.update_from_config_file(lh)
has_config = True
secret_name = os.getenv("CXG_AWS_SECRET_NAME")
secret_region_name = os.getenv("CXG_AWS_SECRET_REGION_NAME")
if config_file:
region_name = discover_s3_region_name(config_file)
config_location = DataLocator(config_file, region_name)
if config_location.exists():
with config_location.local_handle() as lh:
logging.info(f"Configuration from {config_file}")
app_config.update_from_config_file(lh)
else:
logging.critical(f"Configuration file not found {config_file}")
sys.exit(1)
else:
# no config file specified, try "config.yaml" in the current working directory
config_file = "config.yaml"
config_location = DataLocator(config_file)
if config_location.exists():
with config_location.local_handle() as lh:
logging.info(f"Configuration from {config_file}")
app_config.update_from_config_file(lh)
# config file: second, use the CXG_CONFIG_FILE
config_file = os.getenv("CXG_CONFIG_FILE")
if config_file:
region_name = discover_s3_region_name(config_file)
config_location = DataLocator(config_file, region_name)
if config_location.exists():
with config_location.local_handle() as lh:
logging.info(f"Configuration from {config_file}")
app_config.update_from_config_file(lh)
has_config = True
else:
logging.critical(f"Configuration file not found {config_file}")
sys.exit(1)
if not has_config:
logging.critical("No config file found")
sys.exit(1)
dataroot = os.getenv("CXG_DATAROOT")
if dataroot:
logging.info(f"Configuration from CXG_DATAROOT")
app_config.update(multi_dataset__dataroot=dataroot)
secret_name = os.getenv("CXG_AWS_SECRET_NAME")
if secret_name:
# need to find the secret manager region.
# 1. from CXG_AWS_SECRET_REGION_NAME
# 2. discover from dataroot location (if on s3)
# 3. discover from config file location (if on s3)
secret_region_name = os.getenv("CXG_AWS_SECRET_REGION_NAME")
if secret_region_name is None:
secret_region_name = discover_s3_region_name(app_config.multi_dataset__dataroot)
if not secret_region_name:
logging.error(f"Expected to discover the s3 region name from {app_config.multi_dataset__dataroot}")
secret_region_name = discover_s3_region_name(config_file)
if not secret_region_name:
logging.error(f"Expected to discover the s3 region name from {app_config.multi_dataset__dataroot}")
if not secret_region_name:
logging.error(f"Could not determine the AWS Secret Manager region")
sys.exit(1)
flask_secret_key = get_flask_secret_key(secret_region_name, secret_name)
app_config.update(server__flask_secret_key=flask_secret_key)