Addresses the issue where Gunicorn/uWSGI servers import the gateway
module but never call main(), leaving item_sources empty and causing
the file crawler to fail.
Changes:
- Extract data source initialization into initialize_data_sources()
- Call initialization at module import time for WSGI compatibility
- Add _initialized flag to prevent double initialization
- Simplify main() to delegate to initialize_data_sources()
This ensures data sources are populated when running under WSGI servers
(Gunicorn, uWSGI) while maintaining backward compatibility with the
Flask development server.
Related to GitHub issues #33 and #92
When viewing datasets through the gateway, requests would fail with:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 1
The gateway was copying the accept-encoding header from browser requests
when proxying to cellxgene backend servers. When accept-encoding is
manually set, the Python requests library assumes the caller will handle
decompression and leaves response content compressed.
The cellxgene server responded with zstd-compressed content (magic bytes
28 b5 2f fd), but the gateway attempted to decode this compressed binary
data as UTF-8 text, causing the decode error.
Solution: Remove accept-encoding from the copied headers list in
cache_entry.py. This allows the requests library to automatically handle
compression negotiation and transparently decompress responses (gzip,
deflate, brotli, zstd, etc.).
This is the standard practice when proxying with requests and maintains
all other gateway functionality (URL rewriting, auth, caching, etc.).
Tested:
- Dataset viewing works with compressed responses
- File browser and static assets load correctly
- URL rewriting continues to function properly
Andy