Files
cellxgene/client/src/actions/index.js
T
Bruce Martin eeec842ad0 Restv2 feature branch merge to master (#284)
Move to new REST v0.2 communication between front and back-end.   This is a first cut implementation which is functional, but will need follow-up enhancements for performance, error checking, etc.    Protocol spec is in docs directory.

* Add filtering via indexing

* Using new filter specs

Indexing working

* Added filtering by annotation value

* factor out common methods

* Documentation

* create enum for axis (obs/var)

* Better description for filter's return

* Add boolean to enumerated types

* Augmented enum for scanpy axis

* Create schema for annotations

Based on datatype within scanpy/anndata
+ tests

* remove obsolete schema parse script

* Update rest api to remove old routes and add schema route

* Separate development requirements

* Warning for unsupported datatypes

* include -r requirements.txt in dev

* Merged downcast warnings

* Fixed bug where names were NaNs

Needed to include the index too when creating the series

* Add config endpoint

* Generate app features from CLI selections

* Move features to driver

* Add tests for schema

* Clearer version wording

* python3 version of super

* version from engine to package level

* move features to driver

* Revise layout function to match the new spec

* GET for layout/obs

* PUT Layout (#211)

* PUT Layout

* Csweaver/annotations (#212)


* Update scanpy engine to support the rest v0.2 annotation requests

* GET endpoint for obs annotations + tests

* Documentation

* Test annotations in scanpy engine

* Description for annotation-keys param

* annotation->annotations

* clarified return for annotations

* Use URL query list for annotations fields

* parse_filter parses v0.2 GET filters (#215)

* parse_filter parses v0.2 GET filters

* Don't allow index filters from query params

* Better variable conversion

* Parse filter improvements

- uses default dict
- renamed filter -> query_filter

* Cleanup Tasks (#216)

* Add test_api back into travis build

* Do custom JSON encoding the correct way

* Run cellxgene server in test setup

* Cleanup new tests too

* Option to bind to all interfaces (#225)

app.run("0.0.0.0") instead of app.run("127.0.0.1") binds to all interfaces.

Note: There are comments on the internet that says that the flask server is not up to the task of production serving.  I don't think that such scalability concerns apply here, but I was able to get cellxgene working with twistd relatively easily, and we could switch to that if there are scalability concerns.

Test plan: browsed to <ip>:5005/api/v0.2/config on a different host.

* Add filtering via indexing

* Using new filter specs

Indexing working

* Added filtering by annotation value

* factor out common methods

* Documentation

* create enum for axis (obs/var)

* Better description for filter's return

* Add boolean to enumerated types

* Augmented enum for scanpy axis

* Create schema for annotations

Based on datatype within scanpy/anndata
+ tests

* remove obsolete schema parse script

* Update rest api to remove old routes and add schema route

* Separate development requirements

* Warning for unsupported datatypes

* include -r requirements.txt in dev

* Merged downcast warnings

* Fixed bug where names were NaNs

Needed to include the index too when creating the series

* Add config endpoint

* Generate app features from CLI selections

* Move features to driver

* Add tests for schema

* Clearer version wording

* python3 version of super

* version from engine to package level

* move features to driver

* Revise layout function to match the new spec

* GET for layout/obs

* PUT Layout (#211)

* PUT Layout

* Csweaver/annotations (#212)


* Update scanpy engine to support the rest v0.2 annotation requests

* GET endpoint for obs annotations + tests

* Documentation

* Test annotations in scanpy engine

* Description for annotation-keys param

* annotation->annotations

* clarified return for annotations

* Use URL query list for annotations fields

* parse_filter parses v0.2 GET filters (#215)

* parse_filter parses v0.2 GET filters

* Don't allow index filters from query params

* Better variable conversion

* Parse filter improvements

- uses default dict
- renamed filter -> query_filter

* Cleanup Tasks (#216)

* Add test_api back into travis build

* Do custom JSON encoding the correct way

* Run cellxgene server in test setup

* Cleanup new tests too

* Option to bind to all interfaces (#225)

app.run("0.0.0.0") instead of app.run("127.0.0.1") binds to all interfaces.

Note: There are comments on the internet that says that the flask server is not up to the task of production serving.  I don't think that such scalability concerns apply here, but I was able to get cellxgene working with twistd relatively easily, and we could switch to that if there are scalability concerns.

Test plan: browsed to <ip>:5005/api/v0.2/config on a different host.

* Fix merge errors

- import warnings was improperly deleted
- scanpy engine tests were totally wrong

* Fix merge error with driver

* PUT /annotations (#235)

* Add query param for annotation name

* fix descriptions, eliminate else clause

* first cut at initial data load on rest 0.2 api

* Annotation var (#248)

* Fix bug strings are always objects in pandas

* Add axis to annotation method

* Add /annotation/var to REST api

* Csweaver/expressiondata (#242)

* Refactor expression method for REST v2

* Add message to QueryStringError

* Fix range filters

* Add GET route for /data

* /data PUT route

* rename expression to data_frame

* clarification of error

* Improve accept type handling

* support all schema types for 0.2 REST API

* remove REST 0.1 code; connect var annotations loading

* config reducer; use config to set data set title; remove obsolete templating code for data set title

* REST 0.2 expression conversion support

* partial port of expression to REST 0.2

*  diffexp (#273)

* Add diffexp method to scanpy

and test

* Minor tweaks to diffexp

Get a minimal working version to unblock FE development

* Fixing things git deleted

* cleanup print statements

* Add index test

* additional, partial REST 0.2 bring up of diffexp

* Ignore unstructured annotations for data (#275)

This is a temp hack, need to figure out how to include data.uns if there is only one gene

* diffexp REST 0.2 port finish

* ignore unstructured annotaitons on all routes except layout

* correctly use varDataCache; maintain state during world rebuild

* correct varDataCache use

* temporarily disable all memoization

* refinements to expression data caching

* clear cell sets upon regraph/reset

* update version of REST to 0.2

* Travis build fixes

- comment out cache import
- fix duplicate test name

* Remove dependency from travis

* clarify semantics of config variables

* move generic action helpers into util
2018-10-01 14:58:46 -07:00

269 lines
7.6 KiB
JavaScript

// jshint esversion: 6
import _ from "lodash";
import * as globals from "../globals";
import { Universe, kvCache } from "../util/stateManager";
import { catchErrorsWrap, doJsonRequest } from "../util/actionHelpers";
const doInitialDataLoad = () =>
catchErrorsWrap(async dispatch => {
dispatch({ type: "initial data load start" });
try {
const requests = _([
"config",
"schema",
"annotations/obs",
"annotations/var",
"layout/obs"
])
.map(r => `${globals.API.prefix}${globals.API.version}${r}`)
.map(url => doJsonRequest(url))
.value();
const results = await Promise.all(requests);
const universe = Universe.createUniverseFromRestV02Response(...results);
dispatch({
type: "configuration load complete",
config: results[0].config
});
dispatch({
type: "initial data load complete (universe exists)",
universe
});
} catch (error) {
dispatch({ type: "initial data load error", error });
}
});
// XXX TODO - this is the old code for doing a regraph. Preserving it solely
// until we port to 0.2 API. The new UX for regraph can't be implemented on
// the 0.1 API (doesn't allow for re-layout on arbitrary sets of cells), so just
// punting for now. See ticket #88
//
//
// /* SELECT */
// const regraph = () => {
// return (dispatch, getState) => {
// dispatch({ type: "regraph started" });
//
// const state = getState();
// const selectedMetadata = {};
//
// _.each(state.controls.categoricalAsBooleansMap, (options, field) => {
// let atLeastOneOptionDeselected = false;
//
// _.each(options, (isActive, option) => {
// if (!isActive) {
// atLeastOneOptionDeselected = true;
// }
// });
//
// if (atLeastOneOptionDeselected) {
// _.each(options, (isActive, option) => {
// if (isActive) {
// if (selectedMetadata[field]) {
// selectedMetadata[field].push(option);
// } else if (!selectedMetadata[field]) {
// selectedMetadata[field] = [option];
// }
// }
// });
// }
// });
//
// let uri = new URI();
// uri.setSearch(selectedMetadata);
// console.log(uri.search(), selectedMetadata);
//
// dispatch(requestCells(uri.search())).then(res => {
// if (res.error) {
// dispatch({ type: "regraph error" });
// } else {
// dispatch({ type: "regraph success" });
// }
// });
// };
// };
const regraph = () => (dispatch, getState) => {
const { universe, world, crossfilter } = getState().controls;
dispatch({
type: "set World to current selection",
universe,
world,
crossfilter
});
};
const resetGraph = () => (dispatch, getState) =>
dispatch({
type: "reset World to eq Universe",
universe: getState().controls.universe
});
/*
Fetch expression vectors for each gene in genes. This is NOT an action
function, but rather a helper to be called from an action helper that
needs expression data.
Transparently utilizes cached data if it is already present.
*/
async function _doRequestExpressionData(dispatch, getState, genes) {
const state = getState();
const { universe } = state.controls;
/* preload data already in cache */
let expressionData = _.transform(genes, (expData, g) => {
const data = kvCache.get(universe.varDataCache, g);
if (data) {
expData[g] = data;
}
}); // --> { gene: data }
/* make a list of genes for which we do not have data */
const genesToFetch = _.filter(genes, g => expressionData[g] === undefined);
dispatch({ type: "expression load start" });
/* Fetch data for any genes not in cache */
if (genesToFetch.length) {
try {
// XXX: TODO - this could be using /data/var rather than /data/obs,
// as that would simplify the transformation in
// convertExpressionRESTv02ToObject
const res = await fetch(
`${globals.API.prefix}${globals.API.version}data/obs`,
{
method: "PUT",
body: JSON.stringify({
filter: {
var: {
annotation_value: [{ name: "name", values: genesToFetch }]
}
}
}),
headers: new Headers({
accept: "application/json",
"Accept-Encoding": "gzip, deflate, br",
"Content-Type": "application/json"
})
}
);
const data = await res.json();
expressionData = {
...expressionData,
...Universe.convertExpressionRESTv02ToObject(universe, data)
};
} catch (error) {
dispatch({ type: "expression load error", error });
throw error; // rethrow
}
}
dispatch({ type: "expression load success", expressionData });
return expressionData;
}
function requestSingleGeneExpressionCountsForColoringPOST(gene) {
return async (dispatch, getState) => {
dispatch({ type: "get single gene expression for coloring started" });
try {
const expressionData = await _doRequestExpressionData(
dispatch,
getState,
[gene]
);
dispatch({
type: "color by expression",
gene,
data: {
[gene]: expressionData[gene]
}
});
} catch (error) {
dispatch({
type: "get single gene expression for coloring error",
error
});
}
};
}
const requestGeneExpressionCountsPOST = genes => async (dispatch, getState) => {
dispatch({ type: "get expression started" });
try {
const expressionData = await _doRequestExpressionData(
dispatch,
getState,
genes
);
return dispatch({
type: "get expression success",
genes,
data: expressionData
});
} catch (error) {
return dispatch({ type: "get expression error", error });
}
};
const requestDifferentialExpression = (set1, set2, num_genes = 10) => async (
dispatch,
getState
) => {
dispatch({ type: "request differential expression started" });
try {
/*
Steps:
1. get the most differentially expressed genes
2. get expression data for each
*/
const state = getState();
const { universe } = state.controls;
const set1ByIndex = _.map(set1, s => universe.obsNameToIndexMap[s]);
const set2ByIndex = _.map(set2, s => universe.obsNameToIndexMap[s]);
const diffExpFetch = await fetch(
`${globals.API.prefix}${globals.API.version}diffexp/obs`,
{
method: "POST",
headers: new Headers({
Accept: "application/json",
"Accept-Encoding": "gzip, deflate, br",
"Content-Type": "application/json"
}),
body: JSON.stringify({
mode: "topN",
count: num_genes,
set1: { filter: { obs: { index: set1ByIndex } } },
set2: { filter: { obs: { index: set2ByIndex } } }
})
}
);
const data = await diffExpFetch.json();
// result is [ [varIdx, ...], ... ]
const topNGenes = _.map(data, r => universe.varAnnotations[r[0]].name);
/*
Kick off secondary action to fetch all of the expression data for the
topN expressed genes.
*/
dispatch(requestGeneExpressionCountsPOST(topNGenes));
/* then send the success case action through */
return dispatch({
type: "request differential expression success",
data
});
} catch (error) {
return dispatch({
type: "request differential expression error",
error
});
}
};
export default {
regraph,
resetGraph,
requestSingleGeneExpressionCountsForColoringPOST,
requestDifferentialExpression,
doInitialDataLoad
};