Clip continuous values based on percentile cutoffs (#672)

* Add numeric inputs for percentiles

* Define initial values for percentile cutoffs in world reducer

* add percentil to crossfilter dimensions

* worldEqUniverse now handles cloned worlds

* add Dataframe.mapColumns

* Wire up handlers for percentile inputs

* World reducer and stateManager know about continuousPercentileMin/Max

* Create world as universe clone (not pointer) to avoid clobbering vals

* Define basic actions for setting continuousPercentileMin/Max

* Under the hood, deal with percentiles between 0 and 1

* Move percentile inputs to visualization settings menu

* Fix padding for undo/redo buttons

* Trigger world rebuild from percentile actions

* BROKEN - pseudocode for clamping dataframe by percentiles upon world rebuild

* fix error handling on clip quantiles; start world clipping implementation

* more unclipped reorg

* rename crossfilter.percentile to quantile

* simplify schema access

* update continuous legend when scale changes

* update color cache when clip changes

* clip obs annotations and var data when clip quantile changes

* use own fromEntries

* fix tests

* stable non-finite float sort/search

* clarify comments

* fix syntax typo

* use new stand-alone clip

* clip expresssion data

* add select tests for non-finite scalars

* basic styles

* clip UI now requires explicit commit

* reset enable/disable accounts for clip percentiles

* better error messages

* fix bug in undo interaction with programatic min brush selection

* small refactoring

* support clipping of int data

* do not perform unnecessary summarizations

* improve caching of dataframe compiled columns

* add percentile precompute to Dataframe.summarize

* use Dataframe.summarize for clip percentiles

* remove obsolete quantile code from corssfilter

* histogram scale and label Y axis, add unclipped X range labels

* layout tweaks

* scatterplot now updates when clip changes

* improve comments

* remove debugging comment

* rework clip number entry validation for usability

* ui tweaks to histogram colors and layout

* enable undo/redo for clip user action

* refine UI on clip value entry

* api cleanup

* update confusing comment

* clarify purpose of isValidDigitKeyEvent

* fix misleading comment

* apply appropriate button-group classes; do not mix span and div

* variable name and comment changes suggested in PR review

* rename sort to sortArray; remove unused and dead code path

* naming changes suggested in PR review

* code review improvements for clarity

* more small changes from PR review

* lint fixes for PR review

* fix spelling error

* clarify that function performs in-place modification of world

* add comment to clarify intent of range operation

* fix bad indents in comments

* clean up __columnsAccessor comments and code

* improve comments around clipPredicate

* field name consistency

* improve comment on quantiles params
This commit is contained in:
Sidney Bell
2019-04-30 16:20:10 -07:00
committed by Bruce Martin
parent 9f10d8095a
commit a08e19bbd0
34 changed files with 1842 additions and 551 deletions
@@ -532,6 +532,42 @@ describe("dataframe factories", () => {
expect(df.rowIndex.keys()).toEqual(dfA.rowIndex.keys());
});
});
describe("mapColumns", () => {
test("identity", () => {
const dfA = Dataframe.Dataframe.create(
[3, 3],
[
new Array(3).fill(0),
new Int16Array(3).fill(99),
new Float64Array(3).fill(1.1)
]
);
const dfB = dfA.mapColumns((col, idx) => {
expect(dfA.icol(idx).asArray()).toBe(col);
return col;
});
expect(dfA).not.toBe(dfB);
expect(dfA.dims).toEqual(dfB.dims);
for (let c = 0; c < dfA.dims[1]; c += 1) {
expect(dfA.icol(c).asArray()).toBe(dfB.icol(c).asArray());
}
});
test("transform", () => {
const dfA = Dataframe.Dataframe.create(
[3, 3],
[new Array(3).fill(0), new Array(3).fill(0), new Array(3).fill(0)]
);
const dfB = dfA.mapColumns(() => {
return new Array(3).fill(1);
});
expect(dfA).not.toBe(dfB);
expect(dfB.iat(0, 0)).toEqual(1);
expect(dfB.iat(0, 1)).toEqual(1);
expect(dfB.iat(0, 2)).toEqual(1);
});
});
});
describe("dataframe col", () => {