Previously, default_item_source was set to a string ("s3" or "local"),
but matching_source() tried to access default_item_source.name, causing:
AttributeError: 'str' object has no attribute 'name'
This bug occurred when source_name=None (single data source configuration)
and has existed since the ItemSource interface was introduced in 2021.
Changes:
- Store the actual ItemSource object reference instead of string name
- Assign to intermediate variables (s3_source, file_source) for clarity
Fixes the error when viewing datasets with a single data source configured.
Addresses the issue where Gunicorn/uWSGI servers import the gateway
module but never call main(), leaving item_sources empty and causing
the file crawler to fail.
Changes:
- Extract data source initialization into initialize_data_sources()
- Call initialization at module import time for WSGI compatibility
- Add _initialized flag to prevent double initialization
- Simplify main() to delegate to initialize_data_sources()
This ensures data sources are populated when running under WSGI servers
(Gunicorn, uWSGI) while maintaining backward compatibility with the
Flask development server.
Related to GitHub issues #33 and #92
When viewing datasets through the gateway, requests would fail with:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 1
The gateway was copying the accept-encoding header from browser requests
when proxying to cellxgene backend servers. When accept-encoding is
manually set, the Python requests library assumes the caller will handle
decompression and leaves response content compressed.
The cellxgene server responded with zstd-compressed content (magic bytes
28 b5 2f fd), but the gateway attempted to decode this compressed binary
data as UTF-8 text, causing the decode error.
Solution: Remove accept-encoding from the copied headers list in
cache_entry.py. This allows the requests library to automatically handle
compression negotiation and transparently decompress responses (gzip,
deflate, brotli, zstd, etc.).
This is the standard practice when proxying with requests and maintains
all other gateway functionality (URL rewriting, auth, caching, etc.).
Tested:
- Dataset viewing works with compressed responses
- File browser and static assets load correctly
- URL rewriting continues to function properly
This adds the flag `GATEWAY_ENABLE_GENE_SETS` to enable support for gene
sets. To simplify implementation, activating this flag also activates
`GATEWAY_ENABLE_ANNOTATIONS`. The gene sets are saved in a file that
has the same name as the annotations `csv` but with `_gene_sets`
appended to the file name (before the extension). This file is hidden
in filecrawler, and the gene sets are loaded when the associated
annotations file is loaded.
If the annotations file is missing, then an Exception is raised.
I have updated one unit test to make it expect
`--disable-gene-sets-save` in the default case (i.e. if
`GATEWAY_ENABLE_ANNOTATIONS = 0`). All units tests pass.
I have updated the README to document `GATEWAY_ENABLE_GENE_SETS`.
Alok Saldanha