mirror of
https://github.com/chanzuckerberg/cellxgene.git
synced 2026-09-19 19:08:11 +08:00
Move to new REST v0.2 communication between front and back-end. This is a first cut implementation which is functional, but will need follow-up enhancements for performance, error checking, etc. Protocol spec is in docs directory. * Add filtering via indexing * Using new filter specs Indexing working * Added filtering by annotation value * factor out common methods * Documentation * create enum for axis (obs/var) * Better description for filter's return * Add boolean to enumerated types * Augmented enum for scanpy axis * Create schema for annotations Based on datatype within scanpy/anndata + tests * remove obsolete schema parse script * Update rest api to remove old routes and add schema route * Separate development requirements * Warning for unsupported datatypes * include -r requirements.txt in dev * Merged downcast warnings * Fixed bug where names were NaNs Needed to include the index too when creating the series * Add config endpoint * Generate app features from CLI selections * Move features to driver * Add tests for schema * Clearer version wording * python3 version of super * version from engine to package level * move features to driver * Revise layout function to match the new spec * GET for layout/obs * PUT Layout (#211) * PUT Layout * Csweaver/annotations (#212) * Update scanpy engine to support the rest v0.2 annotation requests * GET endpoint for obs annotations + tests * Documentation * Test annotations in scanpy engine * Description for annotation-keys param * annotation->annotations * clarified return for annotations * Use URL query list for annotations fields * parse_filter parses v0.2 GET filters (#215) * parse_filter parses v0.2 GET filters * Don't allow index filters from query params * Better variable conversion * Parse filter improvements - uses default dict - renamed filter -> query_filter * Cleanup Tasks (#216) * Add test_api back into travis build * Do custom JSON encoding the correct way * Run cellxgene server in test setup * Cleanup new tests too * Option to bind to all interfaces (#225) app.run("0.0.0.0") instead of app.run("127.0.0.1") binds to all interfaces. Note: There are comments on the internet that says that the flask server is not up to the task of production serving. I don't think that such scalability concerns apply here, but I was able to get cellxgene working with twistd relatively easily, and we could switch to that if there are scalability concerns. Test plan: browsed to <ip>:5005/api/v0.2/config on a different host. * Add filtering via indexing * Using new filter specs Indexing working * Added filtering by annotation value * factor out common methods * Documentation * create enum for axis (obs/var) * Better description for filter's return * Add boolean to enumerated types * Augmented enum for scanpy axis * Create schema for annotations Based on datatype within scanpy/anndata + tests * remove obsolete schema parse script * Update rest api to remove old routes and add schema route * Separate development requirements * Warning for unsupported datatypes * include -r requirements.txt in dev * Merged downcast warnings * Fixed bug where names were NaNs Needed to include the index too when creating the series * Add config endpoint * Generate app features from CLI selections * Move features to driver * Add tests for schema * Clearer version wording * python3 version of super * version from engine to package level * move features to driver * Revise layout function to match the new spec * GET for layout/obs * PUT Layout (#211) * PUT Layout * Csweaver/annotations (#212) * Update scanpy engine to support the rest v0.2 annotation requests * GET endpoint for obs annotations + tests * Documentation * Test annotations in scanpy engine * Description for annotation-keys param * annotation->annotations * clarified return for annotations * Use URL query list for annotations fields * parse_filter parses v0.2 GET filters (#215) * parse_filter parses v0.2 GET filters * Don't allow index filters from query params * Better variable conversion * Parse filter improvements - uses default dict - renamed filter -> query_filter * Cleanup Tasks (#216) * Add test_api back into travis build * Do custom JSON encoding the correct way * Run cellxgene server in test setup * Cleanup new tests too * Option to bind to all interfaces (#225) app.run("0.0.0.0") instead of app.run("127.0.0.1") binds to all interfaces. Note: There are comments on the internet that says that the flask server is not up to the task of production serving. I don't think that such scalability concerns apply here, but I was able to get cellxgene working with twistd relatively easily, and we could switch to that if there are scalability concerns. Test plan: browsed to <ip>:5005/api/v0.2/config on a different host. * Fix merge errors - import warnings was improperly deleted - scanpy engine tests were totally wrong * Fix merge error with driver * PUT /annotations (#235) * Add query param for annotation name * fix descriptions, eliminate else clause * first cut at initial data load on rest 0.2 api * Annotation var (#248) * Fix bug strings are always objects in pandas * Add axis to annotation method * Add /annotation/var to REST api * Csweaver/expressiondata (#242) * Refactor expression method for REST v2 * Add message to QueryStringError * Fix range filters * Add GET route for /data * /data PUT route * rename expression to data_frame * clarification of error * Improve accept type handling * support all schema types for 0.2 REST API * remove REST 0.1 code; connect var annotations loading * config reducer; use config to set data set title; remove obsolete templating code for data set title * REST 0.2 expression conversion support * partial port of expression to REST 0.2 * diffexp (#273) * Add diffexp method to scanpy and test * Minor tweaks to diffexp Get a minimal working version to unblock FE development * Fixing things git deleted * cleanup print statements * Add index test * additional, partial REST 0.2 bring up of diffexp * Ignore unstructured annotations for data (#275) This is a temp hack, need to figure out how to include data.uns if there is only one gene * diffexp REST 0.2 port finish * ignore unstructured annotaitons on all routes except layout * correctly use varDataCache; maintain state during world rebuild * correct varDataCache use * temporarily disable all memoization * refinements to expression data caching * clear cell sets upon regraph/reset * update version of REST to 0.2 * Travis build fixes - comment out cache import - fix duplicate test name * Remove dependency from travis * clarify semantics of config variables * move generic action helpers into util
103 lines
3.5 KiB
Python
103 lines
3.5 KiB
Python
from abc import ABCMeta, abstractmethod
|
|
|
|
|
|
class CXGDriver(metaclass=ABCMeta):
|
|
|
|
def __init__(self, data, layout_method=None, diffexp_method=None):
|
|
self.data = self._load_data(data)
|
|
self.layout_method = layout_method
|
|
self.diffexp_method = diffexp_method
|
|
self.cluster = None
|
|
|
|
@property
|
|
def features(self):
|
|
features = {
|
|
"cluster": {"available": False},
|
|
"layout": {
|
|
"obs": {"available": False},
|
|
"var": {"available": False},
|
|
},
|
|
"diffexp": {"available": False}
|
|
}
|
|
# TODO - Interactive limit should be generated from the actual available methods see GH issue #94
|
|
if self.layout_method:
|
|
# TODO handle "var" when gene layout becomes available
|
|
features["layout"]["obs"] = {"available": True, "interactiveLimit": 15000}
|
|
if self.diffexp_method:
|
|
features["diffexp"] = {"available": True, "interactiveLimit": 5000}
|
|
if self.cluster:
|
|
features["cluster"] = {"available": True, "interactiveLimit": 45000}
|
|
return features
|
|
|
|
@staticmethod
|
|
@abstractmethod
|
|
def _load_data(data):
|
|
pass
|
|
|
|
@abstractmethod
|
|
def cells(self):
|
|
pass
|
|
|
|
@abstractmethod
|
|
def genes(self):
|
|
pass
|
|
|
|
@abstractmethod
|
|
def filter_dataframe(self, filter):
|
|
"""
|
|
Filter cells from data and return a subset of the data. They can operate on both obs and var dimension with
|
|
indexing and filtering by annotation value. Filters are combined with the and operator.
|
|
See REST specs for info on filter format:
|
|
https://docs.google.com/document/d/1Fxjp1SKtCk7l8QP9-7KAjGXL0eldi_qEnNT0NmlGzXI/edit#heading=h.8qc9q57amldx
|
|
|
|
:param filter: dictionary with filter parames
|
|
:return: View into scanpy object with cells/genes filtered
|
|
"""
|
|
pass
|
|
|
|
@abstractmethod
|
|
def annotation(self, df, axis, fields=None):
|
|
"""
|
|
Gets annotation value for each observation
|
|
|
|
:param axis:
|
|
:param df: from filter_cells, dataframe
|
|
:param fields: list of keys for annotation to return, returns all annotation values if not set.
|
|
:return: dict: names - list of fields in order, data - list of lists or metadata [idx, val1, val2...]
|
|
"""
|
|
pass
|
|
|
|
@abstractmethod
|
|
def layout(self, df):
|
|
"""
|
|
Computes a n-d layout for cells through dimensionality reduction.
|
|
:param df: from filter_cells, dataframe
|
|
:return: [cellid, x, y]
|
|
"""
|
|
pass
|
|
|
|
@abstractmethod
|
|
def diffexp(self, df1, df2, genes):
|
|
"""
|
|
Computes the top differentially expressed variables between two observation sets. If dataframes
|
|
contain a subset of variables, then statistics for all variables will be returned, otherwise
|
|
only the top N vars will be returned.
|
|
:param df1: from filter_cells, dataframe containing first set of observations
|
|
:param df2: from filter_cells, dataframe containing second set of observations
|
|
:param topN: Limit results to top N (Top var mode only)
|
|
:return: top genes, stats and expression values for variables
|
|
"""
|
|
pass
|
|
|
|
@abstractmethod
|
|
def data_frame(self, df):
|
|
"""
|
|
Retrieves expression for each gene for cells in data frame
|
|
:param df: from filter_cells, dataframe
|
|
:return: {
|
|
"var": list of variable ids,
|
|
"obs": [cellid, var1 expression, var2 expression, ...],
|
|
}
|
|
"""
|
|
pass
|