s3 region should have a single config param (#1386)

* s3 region should have a single config param

The s3 region can also now be automatically determined to further
reduce errors.

This patch also fixes a bug with order of handling the config params.
The tiledb config needs to be fixed before attempting to load
(need to handle_adaptor before handle_single_dataset)
This commit is contained in:
bmccandless
2020-04-11 15:20:01 -07:00
committed by GitHub
parent b3c26e7bc7
commit 5dcfcb5467
3 changed files with 37 additions and 8 deletions
+8 -2
View File
@@ -68,14 +68,20 @@ diffexp:
data_locator:
s3:
region_name: us-east-1
# s3 region name.
# if true, then the s3 location is automatically determined from the datapath or dataroot.
# if false/null, then do not set.
# if a string, then use that value (e.g. us-east-1).
region_name: true
adaptor:
cxg_adaptor:
# The key/values under tiledb_ctx will be used to initialize the tiledb Context.
# If 'vfs.s3.region' is not set, then it will automatically use the setting from
# data_locator / s3 / region_name.
tiledb_ctx:
sm.tile_cache_size: 8589934592
sm.num_reader_threads: 32
vfs.s3.region: us-east-1
anndata_adaptor:
backed: false