fork/cellxgene
1
0
Fork 0
mirror of https://github.com/chanzuckerberg/cellxgene.git synced 2026-09-30 01:48:12 +08:00
Code Issues Packages Projects Releases Wiki Activity

sparse column shift encoding. (#1502)

Browse Source 
Many of our matrices are log normalized, which tends to eliminate
the number of non zero values (if there were any).  This prevents
the matrix from being stored as a sparse matrix.  The solution here
is to use a simple transformation to make it sparse again.  The most
common value from each column is subtracted from that column.  These
values that were subtracted are saved in an array called X_col_shift.

The cellxgene code needs to understand how to undo the transformation when
operating over the X matrix.

- added script to create a synthetic dataset for testing
- added a script to convert an existing CXG dataset to a sparse CXG dataset
This commit is contained in:
bmccandless
2020-06-02 08:23:52 -07:00
committed by GitHub
parent d0577b94af
commit 76523d4f32
8 changed files with 297 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files
server/compute/diffexp_cxg.py
+6
View File
@@ -107,6 +107,12 @@ def diffexp_ttest(adaptor, maskA, maskB, top_n=8, diffexp_lfc_cutoff=0.01):
future.cancel()
raise ComputeError(str(e))
if is_sparse:
if adaptor.has_array("X_col_shift"):
X_col_shift = adaptor.open_array("X_col_shift")[:]
meanA += X_col_shift
meanB += X_col_shift
r = diffexp_ttest_from_mean_var(
meanA.astype(dtype),
varA.astype(dtype),