1510-smoke-test (#1548)

* 1510-smoke-test

* config default

* update tests

* update test config

* fix linter errors

* more comments

* address comments

* use npm install in push_tests.yml

* use environment.default.json

* adding docs

* Take care of @mweiden's nits

* Save screenshots in the __tests__/screenshots/ directory

* typo

* docs

* Add chart tests (#1580)

* merge tests

* check if bin creation returned null before rendering charts (#1576)

* check if bin creation returned null before rendering charts

* refactor chart rendering into functions (#1577)

* little fixes from PR

* reintroduce fix to check for null values

* change getAllByClass to return element

* slice instead

* new stackedbar test

* feedback-1573-test (#1579)

* feedback-1573-test

* enable whole test set

* revert tests

Co-authored-by: Timmy Huang <tihuan@users.noreply.github.com>

* tweak test to actually render chart

* include snapshot

* remove async

* fix getAllHistograms

* properly grab id

Co-authored-by: Timmy Huang <tihuan@users.noreply.github.com>

Co-authored-by: Matt Weiden <538456+mweiden@users.noreply.github.com>
Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com>
This commit is contained in:
Timmy Huang
2020-06-24 11:45:39 -07:00
committed by GitHub
parent e22e671f10
commit 83376627e8
42 changed files with 1388 additions and 887 deletions

View File

@@ -4,7 +4,7 @@ on:
push:
branches: master
pull_request:
branches: '*'
branches: "*"
env:
JEST_ENV: prod
@@ -32,7 +32,7 @@ jobs:
run: |
pip install flake8
cd client
npm i "eslint" "eslint-config-airbnb" "eslint-config-prettier" "eslint-loader" "eslint-plugin-filenames" "eslint-plugin-import" "eslint-plugin-jest" "eslint-plugin-jsx-a11y" "eslint-plugin-react" "eslint-plugin-react-hooks" "eslint-plugin-prettier"
npm install
- name: Lint with flake8
run: |
make lint-server
@@ -41,7 +41,6 @@ jobs:
run: |
make lint
unit-test:
runs-on: ubuntu-latest
steps:

9
.gitignore vendored
View File

@@ -19,11 +19,7 @@ venv/
cellxgene/
# client build
server/common/web/static/css/
server/common/web/static/img/
server/common/web/static/media/
server/common/web/static/fonts/
server/common/web/static/js/
server/common/web/static/*
server/common/web/templates/
server/common/web/csp-hashes.json
@@ -53,3 +49,6 @@ docs/_site/
docs/Gemfile.lock
client/.eslintcache
# E2E Testing
ignoreE2E*

View File

@@ -7,6 +7,7 @@ ANNOTATIONS_FILENAME := $(shell basename $(ANNOTATIONS))
.PHONY: clean
clean:
rm -rf node_modules
rm -f __tests__/screenshots/*.png
.PHONY: ci
ci:
@@ -21,39 +22,20 @@ WEBPACK_CONFIG ?= configuration/webpack/webpack.config.prod.js
build:
npm run build $(WEBPACK_CONFIG)
# Formatting code
.PHONY: lint
lint:
npx eslint ./src/
# Development convenience methods
.PHONY: start-frontend
start-frontend:
node server/development.js
.PHONY: e2e
e2e:
node node_modules/jest/bin/jest.js \
--verbose false \
--config __tests__/e2e/e2eJestConfig.json \
e2e/e2e.test.js
.PHONY: e2e-annotations
e2e-annotations:
node node_modules/jest/bin/jest.js \
--verbose false \
--config __tests__/e2e/e2eJestConfig.json \
e2e/e2eAnnotations.test.js
# start an instance of cellxgene and run the end-to-end tests
.PHONY: smoke-test
smoke-test:
start_server_and_test \
'CXG_OPTIONS="--disable-annotations" $(MAKE) start-server' \
$(CXG_SERVER_PORT) \
'$(MAKE) e2e'
'CXG_URL_BASE="http://localhost:$(CXG_SERVER_PORT)" npm run e2e -- --verbose false'
# start an instance of cellxgene and run the end-to-end annotations tests
.PHONY: smoke-test-annotations
smoke-test-annotations:
$(eval TMP_DIR := $(shell mktemp -d /tmp/cellxgene_XXXXXX))
@@ -61,13 +43,12 @@ smoke-test-annotations:
start_server_and_test \
'CXG_OPTIONS="--annotations-file $(TMP_DIR)/$(ANNOTATIONS_FILENAME)" $(MAKE) start-server' \
$(CXG_SERVER_PORT) \
'$(MAKE) e2e-annotations'
'CXG_URL_BASE="http://localhost:$(CXG_SERVER_PORT)" npm run e2e-annotations -- --verbose false'
rm -rf $(TMP_DIR)
.PHONY: unit-test
unit-test:
node node_modules/jest/bin/jest.js \
--testPathIgnorePatterns e2e
node node_modules/jest/bin/jest.js --testPathIgnorePatterns e2e
# pass remaining commands through to npm run
%:

View File

@@ -2,4 +2,4 @@
exports[`did launch page launched 1`] = `"<span style=\\"width: 185px; display: flex; overflow: hidden; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">pbm</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">c3k</span><span style=\\"position: absolute; right: 0px; color: inherit;\\">c3k</span></span></span>"`;
exports[`metadata loads categories and values from dataset appear 1`] = `"<div style=\\"display: flex; justify-content: space-between; align-items: baseline;\\"><div style=\\"display: flex; justify-content: flex-start; align-items: flex-start;\\"><label class=\\"bp3-control bp3-checkbox\\" for=\\"category-select-louvain\\"><input id=\\"category-select-louvain\\" data-testclass=\\"category-select\\" data-testid=\\"louvain:category-select\\" type=\\"checkbox\\" checked=\\"\\"><span class=\\"bp3-control-indicator\\"></span></label><span role=\\"menuitem\\" tabindex=\\"0\\" data-testid=\\"louvain:category-expand\\" style=\\"cursor: pointer;\\"><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><span data-testid=\\"louvain:category-label\\" aria-label=\\"louvain\\" class=\\"\\" tabindex=\\"0\\" style=\\"max-width: 265px;\\"><span style=\\"max-width: 265px; display: flex; overflow: hidden; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">lou</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">vain</span><span style=\\"position: absolute; right: 0px; color: inherit;\\">vain</span></span></span></span></span></span><svg stroke=\\"currentColor\\" fill=\\"currentColor\\" stroke-width=\\"0\\" viewBox=\\"0 0 320 512\\" data-testclass=\\"category-expand-is-not-expanded\\" height=\\"1em\\" width=\\"1em\\" xmlns=\\"http://www.w3.org/2000/svg\\" style=\\"font-size: 10px; margin-left: 5px;\\"><path d=\\"M285.476 272.971L91.132 467.314c-9.373 9.373-24.569 9.373-33.941 0l-22.667-22.667c-9.357-9.357-9.375-24.522-.04-33.901L188.505 256 34.484 101.255c-9.335-9.379-9.317-24.544.04-33.901l22.667-22.667c9.373-9.373 24.569-9.373 33.941 0L285.475 239.03c9.373 9.372 9.373 24.568.001 33.941z\\"></path></svg></span></div><div><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><a role=\\"button\\" data-testclass=\\"colorby\\" data-testid=\\"colorby-louvain\\" class=\\"bp3-button\\" tabindex=\\"0\\"><span icon=\\"tint\\" class=\\"bp3-icon bp3-icon-tint\\"><svg data-icon=\\"tint\\" width=\\"16\\" height=\\"16\\" viewBox=\\"0 0 16 16\\"><desc>tint</desc><path d=\\"M7.88 1s-4.9 6.28-4.9 8.9c.01 2.82 2.34 5.1 4.99 5.1 2.65-.01 5.03-2.3 5.03-5.13C12.99 7.17 7.88 1 7.88 1z\\" fill-rule=\\"evenodd\\"></path></svg></span></a></span></span></div></div><div style=\\"margin-left: 26px;\\"><div></div></div><div></div>"`;
exports[`metadata loads categories and values from dataset appear 1`] = `"<div style=\\"display: flex; justify-content: space-between; align-items: baseline;\\"><div style=\\"display: flex; justify-content: flex-start; align-items: flex-start;\\"><label class=\\"bp3-control bp3-checkbox\\" for=\\"category-select-louvain\\"><input id=\\"category-select-louvain\\" data-testclass=\\"category-select\\" data-testid=\\"louvain:category-select\\" type=\\"checkbox\\" checked=\\"\\"><span class=\\"bp3-control-indicator\\"></span></label><span role=\\"menuitem\\" tabindex=\\"0\\" data-testclass=\\"category-expand\\" data-testid=\\"louvain:category-expand\\" style=\\"cursor: pointer;\\"><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><span data-testid=\\"louvain:category-label\\" aria-label=\\"louvain\\" class=\\"\\" tabindex=\\"0\\" style=\\"max-width: 265px;\\"><span style=\\"max-width: 265px; display: flex; overflow: hidden; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">lou</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">vain</span><span style=\\"position: absolute; right: 0px; color: inherit;\\">vain</span></span></span></span></span></span><svg stroke=\\"currentColor\\" fill=\\"currentColor\\" stroke-width=\\"0\\" viewBox=\\"0 0 320 512\\" data-testclass=\\"category-expand-is-not-expanded\\" height=\\"1em\\" width=\\"1em\\" xmlns=\\"http://www.w3.org/2000/svg\\" style=\\"font-size: 10px; margin-left: 5px;\\"><path d=\\"M285.476 272.971L91.132 467.314c-9.373 9.373-24.569 9.373-33.941 0l-22.667-22.667c-9.357-9.357-9.375-24.522-.04-33.901L188.505 256 34.484 101.255c-9.335-9.379-9.317-24.544.04-33.901l22.667-22.667c9.373-9.373 24.569-9.373 33.941 0L285.475 239.03c9.373 9.372 9.373 24.568.001 33.941z\\"></path></svg></span></div><div><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><a role=\\"button\\" data-testclass=\\"colorby\\" data-testid=\\"colorby-louvain\\" class=\\"bp3-button\\" tabindex=\\"0\\"><span icon=\\"tint\\" class=\\"bp3-icon bp3-icon-tint\\"><svg data-icon=\\"tint\\" width=\\"16\\" height=\\"16\\" viewBox=\\"0 0 16 16\\"><desc>tint</desc><path d=\\"M7.88 1s-4.9 6.28-4.9 8.9c.01 2.82 2.34 5.1 4.99 5.1 2.65-.01 5.03-2.3 5.03-5.13C12.99 7.17 7.88 1 7.88 1z\\" fill-rule=\\"evenodd\\"></path></svg></span></a></span></span></div></div><div style=\\"margin-left: 26px;\\"><div></div></div><div></div>"`;

View File

@@ -0,0 +1,15 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP
exports[`annotations stacked bar graph renders 1`] = `
Array [
"<div data-flip-config=\\"{&quot;translate&quot;:true,&quot;scale&quot;:true,&quot;opacity&quot;:true}\\" data-portal-key=\\"portal\\" class=\\"categorical__value___2RKaC\\" data-testclass=\\"categorical-row\\" style=\\"padding: 4px 0px 4px 7px; display: flex; align-items: baseline; justify-content: space-between; margin-bottom: 2px; border-radius: 2px;\\"><div style=\\"margin: 0px; padding: 0px; user-select: none; width: 220px; display: flex; justify-content: space-between;\\"><div style=\\"display: flex; align-items: baseline;\\"><label for=\\"value-toggle-checkbox-TEST-LABEL\\" class=\\"bp3-control bp3-checkbox\\" style=\\"margin: 0px;\\"><input id=\\"value-toggle-checkbox-TEST-LABEL\\" data-testclass=\\"categorical-value-select\\" data-testid=\\"categorical-value-select-TEST-CATEGORY-TEST-LABEL\\" type=\\"checkbox\\" checked=\\"\\"><span class=\\"bp3-control-indicator\\"></span></label><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><span data-testid=\\"categorical-value-TEST-CATEGORY-TEST-LABEL\\" data-testclass=\\"categorical-value\\" aria-label=\\"TEST-LABEL\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 63px; color: black; font-style: normal; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 63px; color: black; font-style: normal; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">TEST-</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">LABEL</span><span style=\\"position: absolute; right: 0px; color: black;\\">LABEL</span></span></span></span></span></span></div><span style=\\"flex-shrink: 0;\\"></span></div><div><span><span data-testclass=\\"categorical-value-count\\" data-testid=\\"categorical-value-count-TEST-CATEGORY-TEST-LABEL\\" style=\\"color: black;\\">0</span><svg display=\\"none\\" style=\\"margin-left: 5px; width: 11px; height: 11px; background-color: inherit;\\"></svg><span><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><button type=\\"button\\" data-testclass=\\"seeActions\\" data-testid=\\"TEST-CATEGORY:TEST-LABEL:see-actions\\" class=\\"bp3-button bp3-minimal bp3-small\\" tabindex=\\"0\\" style=\\"margin-left: 2px; position: relative; top: -1px; min-height: 16px;\\"><span icon=\\"more\\" class=\\"bp3-icon bp3-icon-more\\"><svg data-icon=\\"more\\" width=\\"10\\" height=\\"10\\" viewBox=\\"0 0 16 16\\"><desc>more</desc><path d=\\"M2 6.03a2 2 0 100 4 2 2 0 100-4zM14 6.03a2 2 0 100 4 2 2 0 100-4zM8 6.03a2 2 0 100 4 2 2 0 100-4z\\" fill-rule=\\"evenodd\\"></path></svg></span></button></span></span></span></span></div></div>",
"<div data-flip-config=\\"{&quot;translate&quot;:true,&quot;scale&quot;:true,&quot;opacity&quot;:true}\\" data-flip-id=\\"1\\" data-portal-key=\\"portal\\" class=\\"categorical__value___2RKaC\\" data-testclass=\\"categorical-row\\" style=\\"padding: 4px 0px 4px 7px; display: flex; align-items: baseline; justify-content: space-between; margin-bottom: 2px; border-radius: 2px;\\"><div style=\\"margin: 0px; padding: 0px; user-select: none; width: 220px; display: flex; justify-content: space-between;\\"><div style=\\"display: flex; align-items: baseline;\\"><label for=\\"value-toggle-checkbox-unassigned\\" class=\\"bp3-control bp3-checkbox\\" style=\\"margin: 0px;\\"><input id=\\"value-toggle-checkbox-unassigned\\" data-testclass=\\"categorical-value-select\\" data-testid=\\"categorical-value-select-TEST-CATEGORY-unassigned\\" type=\\"checkbox\\" checked=\\"\\"><span class=\\"bp3-control-indicator\\"></span></label><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><span data-testid=\\"categorical-value-TEST-CATEGORY-unassigned\\" data-testclass=\\"categorical-value\\" aria-label=\\"unassigned\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 63px; color: rgb(171, 171, 171); font-style: italic; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 63px; color: rgb(171, 171, 171); font-style: italic; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">unass</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">igned</span><span style=\\"position: absolute; right: 0px; color: rgb(171, 171, 171);\\">igned</span></span></span></span></span></span></div><span style=\\"flex-shrink: 0;\\"><canvas class=\\"bp3-popover-targer\\" width=\\"100\\" height=\\"11\\" style=\\"margin-right: 5px; width: 100px; height: 11px;\\"></canvas></span></div><div><span><span data-testclass=\\"categorical-value-count\\" data-testid=\\"categorical-value-count-TEST-CATEGORY-unassigned\\" style=\\"color: rgb(171, 171, 171); font-style: italic;\\">2132</span><svg display=\\"none\\" style=\\"margin-left: 5px; width: 11px; height: 11px; background-color: inherit;\\"></svg><span><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><button type=\\"button\\" data-testclass=\\"seeActions\\" data-testid=\\"TEST-CATEGORY:unassigned:see-actions\\" class=\\"bp3-button bp3-minimal bp3-small\\" tabindex=\\"0\\" style=\\"margin-left: 2px; position: relative; top: -1px; min-height: 16px;\\"><span icon=\\"more\\" class=\\"bp3-icon bp3-icon-more\\"><svg data-icon=\\"more\\" width=\\"10\\" height=\\"10\\" viewBox=\\"0 0 16 16\\"><desc>more</desc><path d=\\"M2 6.03a2 2 0 100 4 2 2 0 100-4zM14 6.03a2 2 0 100 4 2 2 0 100-4zM8 6.03a2 2 0 100 4 2 2 0 100-4z\\" fill-rule=\\"evenodd\\"></path></svg></span></button></span></span></span></span></div></div>",
]
`;
exports[`annotations stacked bar graph renders 2`] = `
Array [
"<div data-flip-config=\\"{&quot;translate&quot;:true,&quot;scale&quot;:true,&quot;opacity&quot;:true}\\" data-portal-key=\\"portal\\" class=\\"categorical__value___2RKaC\\" data-testclass=\\"categorical-row\\" style=\\"padding: 4px 0px 4px 7px; display: flex; align-items: baseline; justify-content: space-between; margin-bottom: 2px; border-radius: 2px;\\"><div style=\\"margin: 0px; padding: 0px; user-select: none; width: 220px; display: flex; justify-content: space-between;\\"><div style=\\"display: flex; align-items: baseline;\\"><label for=\\"value-toggle-checkbox-TEST-LABEL\\" class=\\"bp3-control bp3-checkbox\\" style=\\"margin: 0px;\\"><input id=\\"value-toggle-checkbox-TEST-LABEL\\" data-testclass=\\"categorical-value-select\\" data-testid=\\"categorical-value-select-TEST-CATEGORY-TEST-LABEL\\" type=\\"checkbox\\" checked=\\"\\"><span class=\\"bp3-control-indicator\\"></span></label><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><span data-testid=\\"categorical-value-TEST-CATEGORY-TEST-LABEL\\" data-testclass=\\"categorical-value\\" aria-label=\\"TEST-LABEL\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 63px; color: black; font-style: normal; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 63px; color: black; font-style: normal; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">TEST-</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">LABEL</span><span style=\\"position: absolute; right: 0px; color: black;\\">LABEL</span></span></span></span></span></span></div><span style=\\"flex-shrink: 0;\\"></span></div><div><span><span data-testclass=\\"categorical-value-count\\" data-testid=\\"categorical-value-count-TEST-CATEGORY-TEST-LABEL\\" style=\\"color: black;\\">0</span><svg display=\\"none\\" style=\\"margin-left: 5px; width: 11px; height: 11px; background-color: inherit;\\"></svg><span><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><button type=\\"button\\" data-testclass=\\"seeActions\\" data-testid=\\"TEST-CATEGORY:TEST-LABEL:see-actions\\" class=\\"bp3-button bp3-minimal bp3-small\\" tabindex=\\"0\\" style=\\"margin-left: 2px; position: relative; top: -1px; min-height: 16px;\\"><span icon=\\"more\\" class=\\"bp3-icon bp3-icon-more\\"><svg data-icon=\\"more\\" width=\\"10\\" height=\\"10\\" viewBox=\\"0 0 16 16\\"><desc>more</desc><path d=\\"M2 6.03a2 2 0 100 4 2 2 0 100-4zM14 6.03a2 2 0 100 4 2 2 0 100-4zM8 6.03a2 2 0 100 4 2 2 0 100-4z\\" fill-rule=\\"evenodd\\"></path></svg></span></button></span></span></span></span></div></div>",
"<div data-flip-config=\\"{&quot;translate&quot;:true,&quot;scale&quot;:true,&quot;opacity&quot;:true}\\" data-flip-id=\\"1\\" data-portal-key=\\"portal\\" class=\\"categorical__value___2RKaC\\" data-testclass=\\"categorical-row\\" style=\\"padding: 4px 0px 4px 7px; display: flex; align-items: baseline; justify-content: space-between; margin-bottom: 2px; border-radius: 2px;\\"><div style=\\"margin: 0px; padding: 0px; user-select: none; width: 220px; display: flex; justify-content: space-between;\\"><div style=\\"display: flex; align-items: baseline;\\"><label for=\\"value-toggle-checkbox-unassigned\\" class=\\"bp3-control bp3-checkbox\\" style=\\"margin: 0px;\\"><input id=\\"value-toggle-checkbox-unassigned\\" data-testclass=\\"categorical-value-select\\" data-testid=\\"categorical-value-select-TEST-CATEGORY-unassigned\\" type=\\"checkbox\\" checked=\\"\\"><span class=\\"bp3-control-indicator\\"></span></label><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><span data-testid=\\"categorical-value-TEST-CATEGORY-unassigned\\" data-testclass=\\"categorical-value\\" aria-label=\\"unassigned\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 63px; color: rgb(171, 171, 171); font-style: italic; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 63px; color: rgb(171, 171, 171); font-style: italic; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">unass</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">igned</span><span style=\\"position: absolute; right: 0px; color: rgb(171, 171, 171);\\">igned</span></span></span></span></span></span></div><span style=\\"flex-shrink: 0;\\"><canvas class=\\"bp3-popover-targer\\" width=\\"100\\" height=\\"11\\" style=\\"margin-right: 5px; width: 100px; height: 11px;\\"></canvas></span></div><div><span><span data-testclass=\\"categorical-value-count\\" data-testid=\\"categorical-value-count-TEST-CATEGORY-unassigned\\" style=\\"color: rgb(171, 171, 171); font-style: italic;\\">2638</span><svg display=\\"none\\" style=\\"margin-left: 5px; width: 11px; height: 11px; background-color: inherit;\\"></svg><span><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><button type=\\"button\\" data-testclass=\\"seeActions\\" data-testid=\\"TEST-CATEGORY:unassigned:see-actions\\" class=\\"bp3-button bp3-minimal bp3-small\\" tabindex=\\"0\\" style=\\"margin-left: 2px; position: relative; top: -1px; min-height: 16px;\\"><span icon=\\"more\\" class=\\"bp3-icon bp3-icon-more\\"><svg data-icon=\\"more\\" width=\\"10\\" height=\\"10\\" viewBox=\\"0 0 16 16\\"><desc>more</desc><path d=\\"M2 6.03a2 2 0 100 4 2 2 0 100-4zM14 6.03a2 2 0 100 4 2 2 0 100-4zM8 6.03a2 2 0 100 4 2 2 0 100-4z\\" fill-rule=\\"evenodd\\"></path></svg></span></button></span></span></span></span></div></div>",
]
`;

View File

@@ -1,228 +1,315 @@
/* eslint-disable no-await-in-loop -- await in loop is needed to emulate sequential user actions */
import { strict as assert } from "assert";
import {
clearInputAndTypeInto,
clickOn,
getAllByClass,
getOneElementInnerText,
typeInto,
waitByID,
waitByClass,
waitForAllByIds,
clickOnUntil,
getTestClass,
getTestId,
isElementPresent,
} from "./puppeteerUtils";
const cellxgeneActions = (page, utils) => ({
async drag(testId, start, end, lasso = false) {
const layout = await utils.waitByID(testId);
const elBox = await layout.boxModel();
const x1 = elBox.content[0].x + start.x;
const x2 = elBox.content[0].x + end.x;
const y1 = elBox.content[0].y + start.y;
const y2 = elBox.content[0].y + end.y;
export async function drag(testId, start, end, lasso = false) {
const layout = await waitByID(testId);
const elBox = await layout.boxModel();
const x1 = elBox.content[0].x + start.x;
const x2 = elBox.content[0].x + end.x;
const y1 = elBox.content[0].y + start.y;
const y2 = elBox.content[0].y + end.y;
await page.mouse.move(x1, y1);
await page.mouse.down();
if (lasso) {
await page.mouse.move(x2, y1);
await page.mouse.move(x2, y2);
await page.mouse.move(x1, y2);
await page.mouse.move(x1, y1);
await page.mouse.down();
if (lasso) {
await page.mouse.move(x2, y1);
await page.mouse.move(x2, y2);
await page.mouse.move(x1, y2);
await page.mouse.move(x1, y1);
} else {
await page.mouse.move(x2, y2);
}
await page.mouse.up();
},
} else {
await page.mouse.move(x2, y2);
}
await page.mouse.up();
}
async clickOnCoordinate(testId, coord) {
const layout = await utils.waitByID(testId);
const elBox = await layout.boxModel();
const x = elBox.content[0].x + coord.x;
const y = elBox.content[0].y + coord.y;
await page.mouse.click(x, y);
},
export async function clickOnCoordinate(testId, coord) {
const layout = await expect(page).toMatchElement(getTestId(testId));
const elBox = await layout.boxModel();
async getAllHistograms(testclass, testIds) {
const histTestIds = testIds.map((tid) => `histogram-${tid}`);
// these load asynchronously, so we need to wait for each histogram individually,
// and they may be quite slow in some cases.
await utils.waitForAllByIds(histTestIds, { timeout: 240000 });
const allHistograms = await utils.getAllByClass(testclass);
return allHistograms.map((hist) => hist.replace(/^histogram-/, ""));
},
if (!elBox) {
throw Error("Layout's boxModel is not available!");
}
async getAllCategoriesAndCounts(category) {
// these load asynchronously, so we have to wait for the specific category.
await utils.waitByID(`category-${category}`);
return page.$$eval(
`[data-testid="category-${category}"] [data-testclass='categorical-row']`,
(rows) =>
Object.fromEntries(
rows.map((row) => {
const cat = row
.querySelector("[data-testclass='categorical-value']")
.getAttribute("aria-label");
const x = elBox.content[0].x + coord.x;
const y = elBox.content[0].y + coord.y;
await page.mouse.click(x, y);
}
const count = row.querySelector(
"[data-testclass='categorical-value-count']"
).innerText;
return [cat, count];
})
)
export async function getAllHistograms(testclass, testIds) {
const histTestIds = testIds.map((tid) => `histogram-${tid}`);
// these load asynchronously, so we need to wait for each histogram individually,
// and they may be quite slow in some cases.
await waitForAllByIds(histTestIds, { timeout: 4 * 60 * 1000 });
const allHistograms = await getAllByClass(testclass);
const testIDs = await Promise.all(
allHistograms.map((hist) => {
return page.evaluate((elem) => {
return elem.dataset.testid;
}, hist);
})
);
return testIDs.map((id) => id.replace(/^histogram-/, ""));
}
export async function getAllCategoriesAndCounts(category) {
// these load asynchronously, so we have to wait for the specific category.
await waitByID(`category-${category}`);
return page.$$eval(
`[data-testid="category-${category}"] [data-testclass='categorical-row']`,
(rows) =>
Object.fromEntries(
rows.map((row) => {
const cat = row
.querySelector("[data-testclass='categorical-value']")
.getAttribute("aria-label");
const count = row.querySelector(
"[data-testclass='categorical-value-count']"
).innerText;
return [cat, count];
})
)
);
}
export async function getCellSetCount(num) {
await clickOn(`cellset-button-${num}`);
return getOneElementInnerText(`[data-testid='cellset-count-${num}']`);
}
export async function resetCategory(category) {
const checkboxId = `${category}:category-select`;
await waitByID(checkboxId);
const checkedPseudoclass = await page.$eval(
`[data-testid='${checkboxId}']`,
(el) => el.matches(":checked")
);
if (!checkedPseudoclass) await clickOn(checkboxId);
const categoryRow = await waitByID(`${category}:category-expand`);
const isExpanded = await categoryRow.$(
"[data-testclass='category-expand-is-expanded']"
);
if (isExpanded) await clickOn(`${category}:category-expand`);
}
export async function calcCoordinate(testId, xAsPercent, yAsPercent) {
const el = await waitByID(testId);
const size = await el.boxModel();
return {
x: Math.floor(size.width * xAsPercent),
y: Math.floor(size.height * yAsPercent),
};
}
export async function calcDragCoordinates(testId, coordinateAsPercent) {
return {
start: await calcCoordinate(
testId,
coordinateAsPercent.x1,
coordinateAsPercent.y1
),
end: await calcCoordinate(
testId,
coordinateAsPercent.x2,
coordinateAsPercent.y2
),
};
}
export async function selectCategory(category, values, reset = true) {
if (reset) await resetCategory(category);
await clickOn(`${category}:category-expand`);
await clickOn(`${category}:category-select`);
for (const value of values) {
await clickOn(`categorical-value-select-${category}-${value}`);
}
}
export async function expandCategory(category) {
const expand = await waitByID(`${category}:category-expand`);
const notExpanded = await expand.$(
"[data-testclass='category-expand-is-not-expanded']"
);
if (notExpanded) await clickOn(`${category}:category-expand`);
}
export async function clip(min = 0, max = 100) {
await clickOn("visualization-settings");
await clearInputAndTypeInto("clip-min-input", min);
await clearInputAndTypeInto("clip-max-input", max);
await clickOn("clip-commit");
}
export async function createCategory(categoryName) {
await clickOnUntil("open-annotation-dialog", async () => {
await expect(page).toMatchElement(getTestId("new-category-name"));
});
await typeInto("new-category-name", categoryName);
await clickOn("submit-category");
}
export async function duplicateCategory(categoryName) {
await clickOn("open-annotation-dialog");
await typeInto("new-category-name", categoryName);
const dropdownOptionClass = "duplicate-category-dropdown-option";
await clickOnUntil("duplicate-category-dropdown", async () => {
await expect(page).toMatchElement(getTestClass(dropdownOptionClass));
});
const option = await expect(page).toMatchElement(
getTestClass(dropdownOptionClass)
);
await option.click();
await clickOnUntil("submit-category", async () => {
await expect(page).toMatchElement(
getTestId(`${categoryName}:category-expand`)
);
},
});
async cellSet(num) {
await utils.clickOn(`cellset-button-${num}`);
return utils.getOneElementInnerText(`[data-testid='cellset-count-${num}']`);
},
await waitByClass("autosave-complete");
}
async resetCategory(category) {
const checkboxId = `${category}:category-select`;
await utils.waitByID(checkboxId);
const checkedPseudoclass = await page.$eval(
`[data-testid='${checkboxId}']`,
(el) => el.matches(":checked")
);
if (!checkedPseudoclass) await utils.clickOn(checkboxId);
try {
const categoryRow = await utils.waitByID(`${category}:category-expand`);
const isExpanded = await categoryRow.$(
"[data-testclass='category-expand-is-expanded']"
);
if (isExpanded) await utils.clickOn(`${category}:category-expand`);
} catch {}
},
export async function renameCategory(oldCategoryName, newCategoryName) {
await clickOn(`${oldCategoryName}:see-actions`);
await clickOn(`${oldCategoryName}:edit-category-mode`);
await clearInputAndTypeInto(
`${oldCategoryName}:edit-category-name-text`,
newCategoryName
);
await clickOn(`${oldCategoryName}:submit-category-edit`);
}
async calcCoordinate(testId, xAsPercent, yAsPercent) {
const el = await utils.waitByID(testId);
const size = await el.boxModel();
return {
x: Math.floor(size.width * xAsPercent),
y: Math.floor(size.height * yAsPercent),
};
},
export async function deleteCategory(categoryName) {
const targetId = `${categoryName}:delete-category`;
async calcDragCoordinates(testId, coordinateAsPercent) {
return {
start: await this.calcCoordinate(
testId,
coordinateAsPercent.x1,
coordinateAsPercent.y1
),
end: await this.calcCoordinate(
testId,
coordinateAsPercent.x2,
coordinateAsPercent.y2
),
};
},
await clickOnUntil(`${categoryName}:see-actions`, async () => {
await expect(page).toMatchElement(getTestId(targetId));
});
async selectCategory(category, values, reset = true) {
if (reset) await this.resetCategory(category);
await utils.clickOn(`${category}:category-expand`);
await utils.clickOn(`${category}:category-select`);
for (const val of values) {
await utils.clickOn(`categorical-value-select-${category}-${val}`);
}
},
await clickOn(targetId);
async expandCategory(category) {
const expand = await utils.waitByID(`${category}:category-expand`);
const notExpanded = await expand.$(
"[data-testclass='category-expand-is-not-expanded']"
);
if (notExpanded) await utils.clickOn(`${category}:category-expand`);
},
await assertCategoryDoesNotExist();
}
async clip(min = 0, max = 100) {
await utils.clickOn("visualization-settings");
await utils.clearInputAndTypeInto("clip-min-input", min);
await utils.clearInputAndTypeInto("clip-max-input", max);
await utils.clickOn("clip-commit");
},
export async function createLabel(categoryName, labelName) {
/**
* (thuang): This explicit wait is needed, since currently showing
* the modal again quickly after the previous action dismissing the
* modal will persist the input value from the previous action.
*
* To reproduce:
* 1. Click on the plus sign to show the modal to add a new label to the category
* 2. Type `123` in the input box
* 3. Hover over your mouse over the plus sign and double click to quickly dismiss and
* invoke the modal again
* 4. You will see `123` is persisted in the input box
* 5. Expected behavior is to get an empty input box
*/
await page.waitFor(500);
async createCategory(categoryName) {
await utils.clickOn("open-annotation-dialog");
await utils.typeInto("new-category-name", categoryName);
await utils.clickOn("submit-category");
},
await clickOn(`${categoryName}:see-actions`);
async renameCategory(oldCatgoryName, newCategoryName) {
await utils.clickOn(`${oldCatgoryName}:see-actions`);
await utils.clickOn(`${oldCatgoryName}:edit-category-mode`);
await utils.clearInputAndTypeInto(
`${oldCatgoryName}:edit-category-name-text`,
newCategoryName
);
await utils.clickOn(`${oldCatgoryName}:submit-category-edit`);
},
await clickOn(`${categoryName}:add-new-label-to-category`);
async deleteCategory(categoryName) {
await utils.clickOn(`${categoryName}:see-actions`);
await utils.clickOn(`${categoryName}:delete-category`);
},
await typeInto(`${categoryName}:new-label-name`, labelName);
async createLabel(categoryName, labelName) {
await utils.clickOn(`${categoryName}:see-actions`);
await utils.clickOn(`${categoryName}:add-new-label-to-category`);
await utils.typeInto(`${categoryName}:new-label-name`, labelName);
await utils.clickOn(`${categoryName}:submit-label`);
},
await clickOn(`${categoryName}:submit-label`);
}
async deleteLabel(categoryName, labelName) {
await this.expandCategory(categoryName);
await utils.clickOn(`${categoryName}:${labelName}:see-actions`);
await utils.clickOn(`${categoryName}:${labelName}:delete-label`);
},
export async function deleteLabel(categoryName, labelName) {
await expandCategory(categoryName);
await clickOn(`${categoryName}:${labelName}:see-actions`);
await clickOn(`${categoryName}:${labelName}:delete-label`);
}
async renameLabel(categoryName, oldLabelName, newLabelName) {
await this.expandCategory(categoryName);
await utils.clickOn(`${categoryName}:${oldLabelName}:see-actions`);
await utils.clickOn(`${categoryName}:${oldLabelName}:edit-label`);
await utils.clearInputAndTypeInto(
`${categoryName}:${oldLabelName}:edit-label-name`,
newLabelName
);
await utils.clickOn(`${categoryName}:${oldLabelName}:submit-label-edit`);
},
export async function renameLabel(categoryName, oldLabelName, newLabelName) {
await expandCategory(categoryName);
await clickOn(`${categoryName}:${oldLabelName}:see-actions`);
await clickOn(`${categoryName}:${oldLabelName}:edit-label`);
await clearInputAndTypeInto(
`${categoryName}:${oldLabelName}:edit-label-name`,
newLabelName
);
await clickOn(`${categoryName}:${oldLabelName}:submit-label-edit`);
}
async addGeneToSearch(geneName) {
await utils.typeInto("gene-search", geneName);
await page.keyboard.press("Enter");
await page.waitForSelector(`[data-testid='histogram-${geneName}']`);
},
export async function addGeneToSearch(geneName) {
await typeInto("gene-search", geneName);
await page.keyboard.press("Enter");
await page.waitForSelector(`[data-testid='histogram-${geneName}']`);
}
async subset(coordinatesAsPercent) {
// In order to deselect the selection after the subset, make sure we have some clear part
// of the scatterplot we can click on
assert(coordinatesAsPercent.x2 < 0.99 || coordinatesAsPercent.y2 < 0.99);
const lassoSelection = await this.calcDragCoordinates(
"layout-graph",
coordinatesAsPercent
);
await this.drag(
"layout-graph",
lassoSelection.start,
lassoSelection.end,
true
);
await utils.clickOn("subset-button");
const clearCoordinate = await this.calcCoordinate(
"layout-graph",
0.5,
0.99
);
await this.clickOnCoordinate("layout-graph", clearCoordinate);
},
export async function subset(coordinatesAsPercent) {
// In order to deselect the selection after the subset, make sure we have some clear part
// of the scatterplot we can click on
assert(coordinatesAsPercent.x2 < 0.99 || coordinatesAsPercent.y2 < 0.99);
const lassoSelection = await calcDragCoordinates(
"layout-graph",
coordinatesAsPercent
);
await drag("layout-graph", lassoSelection.start, lassoSelection.end, true);
await clickOn("subset-button");
const clearCoordinate = await calcCoordinate("layout-graph", 0.5, 0.99);
await clickOnCoordinate("layout-graph", clearCoordinate);
}
async setSellSet(cellSet, cellSetNum) {
for (const selection of cellSet.filter(
(sel) => sel.kind === "categorical"
)) {
await this.selectCategory(selection.metadata, selection.values, true);
}
await this.cellSet(cellSetNum);
},
export async function setSellSet(cellSet, cellSetNum) {
const selections = cellSet.filter((sel) => sel.kind === "categorical");
async runDiffExp(cellSet1, cellSet2) {
await this.setSellSet(cellSet1, 1);
await this.setSellSet(cellSet2, 2);
await utils.clickOn("diffexp-button");
},
for (const selection of selections) {
await selectCategory(selection.metadata, selection.values, true);
}
async bulkAddGenes(geneNames) {
await utils.clickOn("section-bulk-add");
await utils.typeInto("input-bulk-add", geneNames.join(","));
await page.keyboard.press("Enter");
},
});
await getCellSetCount(cellSetNum);
}
export default cellxgeneActions;
export async function runDiffExp(cellSet1, cellSet2) {
await setSellSet(cellSet1, 1);
await setSellSet(cellSet2, 2);
await clickOn("diffexp-button");
}
export async function bulkAddGenes(geneNames) {
await clickOn("section-bulk-add");
await typeInto("input-bulk-add", geneNames.join(","));
await page.keyboard.press("Enter");
}
export async function assertCategoryDoesNotExist(categoryName) {
const result = await isElementPresent(
getTestId(`${categoryName}:category-label`)
);
await expect(result).toBe(false);
}
/* eslint-enable no-await-in-loop -- await in loop is needed to emulate sequential user actions */

View File

@@ -1,11 +1,8 @@
export const jestEnv = process.env.JEST_ENV;
export const appPort = process.env.CXG_SERVER_PORT;
export const appUrlBase =
process.env.CXG_URL_BASE || `http://localhost:${appPort}`;
export const DEV = jestEnv === "dev";
export const DEBUG = jestEnv === "debug";
export const DATASET = "pbmc3k";
import * as ENV_DEFAULT from "../../../environment.default.json";
if (DEBUG) jest.setTimeout(2 * 60 * 1000);
if (DEV) jest.setTimeout(30 * 1000);
if (!DEBUG && !DEV) jest.setTimeout(10 * 1000);
export const jestEnv = process.env.JEST_ENV || ENV_DEFAULT.JEST_ENV;
export const appUrlBase =
process.env.CXG_URL_BASE || `http://localhost:${ENV_DEFAULT.CXG_CLIENT_PORT}`;
export const DATASET = "pbmc3k";
export const isDev = jestEnv === ENV_DEFAULT.DEV;
export const isDebug = jestEnv === ENV_DEFAULT.DEBUG;

View File

@@ -27,7 +27,7 @@ export const datasets = {
lasso: [
{
"coordinates-as-percent": { x1: 0.1, y1: 0.25, x2: 0.7, y2: 0.75 },
count: "1173"
count: "1173",
},
],
categorical: [
@@ -111,7 +111,7 @@ export const datasets = {
panzoom: {
lasso: {
"coordinates-as-percent": { x1: 0.3, y1: 0.3, x2: 0.5, y2: 0.5 },
count: "24",
count: "38",
},
},
},

View File

@@ -1,62 +1,69 @@
/*
Smoke test suite that will be run in Travis CI
Tests included in this file are expected to be relatively stable and test core features
/**
* Smoke test suite that will be run in Travis CI
* Tests included in this file are expected to be relatively stable and test core features
*/
import { appUrlBase, DATASET, DEBUG } from "./config";
import setupTestBrowser from "./testBrowser";
/* eslint-disable no-await-in-loop -- await in loop is needed to emulate sequential user actions */
import { appUrlBase, DATASET } from "./config";
import { datasets } from "./data";
let browser;
let page;
let utils;
let cxgActions;
import {
clickOn,
getAllByClass,
getElementCoordinates,
getOneElementInnerHTML,
getTestId,
goToPage,
typeInto,
waitByID,
} from "./puppeteerUtils";
import {
addGeneToSearch,
bulkAddGenes,
calcDragCoordinates,
clip,
drag,
getAllCategoriesAndCounts,
getAllHistograms,
getCellSetCount,
runDiffExp,
selectCategory,
subset,
} from "./cellxgeneActions";
const data = datasets[DATASET];
beforeAll(async () => {
[browser, page, utils, cxgActions] = await setupTestBrowser();
});
beforeEach(async () => {
await page.goto(appUrlBase);
});
afterAll(() => {
if (!DEBUG && browser !== undefined) browser.close();
});
describe("did launch", () => {
test("page launched", async () => {
const element = await utils.getOneElementInnerHTML(
"[data-testid='header']"
);
expect(element).toMatchSnapshot();
});
await goToPage(appUrlBase);
test("terms of service, if they are there", async () => {
try {
await utils.clickOn("tos-cookies-accept", { timeout: 3000 });
} catch {
console.warn("No terms of service footer detected.");
}
page.waitFor(50); // give the footer a chance to disappear
const result = await page.$("[data-testid='tos-cookies-accept']");
expect(result).toBeNull();
const element = await getOneElementInnerHTML(getTestId("header"));
expect(element).toMatchSnapshot();
});
});
describe("metadata loads", () => {
test("categories and values from dataset appear", async () => {
for (const label in data.categorical) {
const elem = await utils.getOneElementInnerHTML(
`[data-testid="category-${label}"]`
await goToPage(appUrlBase);
for (const label of Object.keys(data.categorical)) {
const element = await getOneElementInnerHTML(
getTestId(`category-${label}`)
);
expect(elem).toMatchSnapshot();
await utils.clickOn(`${label}:category-expand`);
const categories = await cxgActions.getAllCategoriesAndCounts(label);
expect(element).toMatchSnapshot();
await clickOn(`${label}:category-expand`);
const categories = await getAllCategoriesAndCounts(label);
expect(Object.keys(categories)).toMatchObject(
Object.keys(data.categorical[label])
);
expect(Object.values(categories)).toMatchObject(
Object.values(data.categorical[label])
);
@@ -64,74 +71,98 @@ describe("metadata loads", () => {
});
test("continuous data appears", async () => {
for (const label in data.continuous) {
await utils.waitByID(`histogram-${label}`);
await goToPage(appUrlBase);
for (const label of Object.keys(data.continuous)) {
await waitByID(`histogram-${label}`);
}
});
});
describe("cell selection", () => {
test("selects all cells cellset 1", async () => {
const cellCount = await cxgActions.cellSet(1);
await goToPage(appUrlBase);
const cellCount = await getCellSetCount(1);
expect(cellCount).toBe(data.dataframe.nObs);
});
test("selects all cells cellset 2", async () => {
const cellCount = await cxgActions.cellSet(2);
await goToPage(appUrlBase);
const cellCount = await getCellSetCount(2);
expect(cellCount).toBe(data.dataframe.nObs);
});
test("selects cells via lasso", async () => {
await goToPage(appUrlBase);
for (const cellset of data.cellsets.lasso) {
const cellset1 = await cxgActions.calcDragCoordinates(
const cellset1 = await calcDragCoordinates(
"layout-graph",
cellset["coordinates-as-percent"]
);
await cxgActions.drag("layout-graph", cellset1.start, cellset1.end, true);
const cellCount = await cxgActions.cellSet(1);
await drag("layout-graph", cellset1.start, cellset1.end, true);
const cellCount = await getCellSetCount(1);
expect(cellCount).toBe(cellset.count);
}
});
test("selects cells via categorical", async () => {
await goToPage(appUrlBase);
for (const cellset of data.cellsets.categorical) {
await utils.clickOn(`${cellset.metadata}:category-expand`);
await utils.clickOn(`${cellset.metadata}:category-select`);
for (const val of cellset.values) {
await utils.clickOn(
`categorical-value-select-${cellset.metadata}-${val}`
);
await clickOn(`${cellset.metadata}:category-expand`);
await clickOn(`${cellset.metadata}:category-select`);
for (const value of cellset.values) {
await clickOn(`categorical-value-select-${cellset.metadata}-${value}`);
}
const cellCount = await cxgActions.cellSet(1);
const cellCount = await getCellSetCount(1);
expect(cellCount).toBe(cellset.count);
}
});
test("selects cells via continuous", async () => {
await goToPage(appUrlBase);
for (const cellset of data.cellsets.continuous) {
const histBrushableAreaId = `histogram-${cellset.metadata}-plot-brushable-area`;
const coords = await cxgActions.calcDragCoordinates(
const coords = await calcDragCoordinates(
histBrushableAreaId,
cellset["coordinates-as-percent"]
);
await cxgActions.drag(histBrushableAreaId, coords.start, coords.end);
const cellCount = await cxgActions.cellSet(1);
await drag(histBrushableAreaId, coords.start, coords.end);
const cellCount = await getCellSetCount(1);
expect(cellCount).toBe(cellset.count);
}
});
});
describe("gene entry", () => {
test("search for single gene", async () =>
cxgActions.addGeneToSearch(data.genes.search));
test("search for single gene", async () => {
await goToPage(appUrlBase);
addGeneToSearch(data.genes.search);
});
test("bulk add genes", async () => {
await goToPage(appUrlBase);
const testGenes = data.genes.bulkadd;
await cxgActions.bulkAddGenes(testGenes);
const allHistograms = await cxgActions.getAllHistograms(
await bulkAddGenes(testGenes);
const allHistograms = await getAllHistograms(
"histogram-user-gene",
testGenes
);
expect(allHistograms).toEqual(expect.arrayContaining(testGenes));
expect(allHistograms).toHaveLength(testGenes.length);
});
@@ -139,31 +170,42 @@ describe("gene entry", () => {
describe("differential expression", () => {
test("selects cells, saves them and performs diffexp", async () => {
await cxgActions.runDiffExp(data.diffexp.cellset1, data.diffexp.cellset2);
const allHistograms = await cxgActions.getAllHistograms(
await goToPage(appUrlBase);
await runDiffExp(data.diffexp.cellset1, data.diffexp.cellset2);
const allHistograms = await getAllHistograms(
"histogram-diffexp",
data.diffexp["gene-results"]
);
expect(allHistograms).toEqual(
expect.arrayContaining(data.diffexp["gene-results"])
);
expect(allHistograms).toHaveLength(data.diffexp["gene-results"].length);
});
});
describe("subset", () => {
test("subset - cell count matches", async () => {
await goToPage(appUrlBase);
for (const select of data.subset.cellset1) {
if (select.kind === "categorical") {
await cxgActions.selectCategory(select.metadata, select.values, true);
await selectCategory(select.metadata, select.values, true);
}
}
await utils.clickOn("subset-button");
for (const label in data.subset.categorical) {
const categories = await cxgActions.getAllCategoriesAndCounts(label);
await clickOn("subset-button");
for (const label of Object.keys(data.subset.categorical)) {
const categories = await getAllCategoriesAndCounts(label);
expect(Object.keys(categories)).toMatchObject(
Object.keys(data.subset.categorical[label])
);
expect(Object.values(categories)).toMatchObject(
Object.values(data.subset.categorical[label])
);
@@ -171,79 +213,99 @@ describe("subset", () => {
});
test("lasso after subset", async () => {
await goToPage(appUrlBase);
for (const select of data.subset.cellset1) {
if (select.kind === "categorical") {
await cxgActions.selectCategory(select.metadata, select.values, true);
await selectCategory(select.metadata, select.values, true);
}
}
await utils.clickOn("subset-button");
const lassoSelection = await cxgActions.calcDragCoordinates(
await clickOn("subset-button");
const lassoSelection = await calcDragCoordinates(
"layout-graph",
data.subset.lasso["coordinates-as-percent"]
);
await cxgActions.drag(
"layout-graph",
lassoSelection.start,
lassoSelection.end,
true
);
const cellCount = await cxgActions.cellSet(1);
await drag("layout-graph", lassoSelection.start, lassoSelection.end, true);
const cellCount = await getCellSetCount(1);
expect(cellCount).toBe(data.subset.lasso.count);
});
test("undo selection appends the top diff exp genes to user defined genes", async () => {
await goToPage(appUrlBase);
const userDefinedGenes = data.genes.bulkadd;
const diffExpGenes = data.diffexp["gene-results"];
await cxgActions.bulkAddGenes(userDefinedGenes);
const userDefinedHistograms = await cxgActions.getAllHistograms(
await bulkAddGenes(userDefinedGenes);
const userDefinedHistograms = await getAllHistograms(
"histogram-user-gene",
userDefinedGenes
);
expect(userDefinedHistograms).toEqual(
expect.arrayContaining(userDefinedGenes)
);
await cxgActions.subset({ x1: 0.15, y1: 0.1, x2: 0.98, y2: 0.98 });
await cxgActions.runDiffExp(data.diffexp.cellset1, data.diffexp.cellset2);
const diffExpHistograms = await cxgActions.getAllHistograms(
await subset({ x1: 0.15, y1: 0.1, x2: 0.98, y2: 0.98 });
await runDiffExp(data.diffexp.cellset1, data.diffexp.cellset2);
const diffExpHistograms = await getAllHistograms(
"histogram-diffexp",
diffExpGenes
);
expect(diffExpHistograms).toEqual(expect.arrayContaining(diffExpGenes));
await utils.clickOn("reset-subset-button");
await clickOn("reset-subset-button");
const expected = [].concat(userDefinedGenes, diffExpGenes);
const userDefinedHistogramsAfterSubset = await cxgActions.getAllHistograms(
const userDefinedHistogramsAfterSubset = await getAllHistograms(
"histogram-user-gene",
expected
);
expect(userDefinedHistogramsAfterSubset).toEqual(
expect.arrayContaining(expected)
);
});
test("subset selection appends the top diff exp genes to user defined genes", async () => {
await goToPage(appUrlBase);
const userDefinedGenes = data.genes.bulkadd;
const diffExpGenes = data.diffexp["gene-results"];
await cxgActions.bulkAddGenes(userDefinedGenes);
const userDefinedHistograms = await cxgActions.getAllHistograms(
await bulkAddGenes(userDefinedGenes);
const userDefinedHistograms = await getAllHistograms(
"histogram-user-gene",
userDefinedGenes
);
expect(userDefinedHistograms).toEqual(
expect.arrayContaining(userDefinedGenes)
);
await cxgActions.subset({ x1: 0.15, y1: 0.1, x2: 0.98, y2: 0.98 });
await cxgActions.runDiffExp(data.diffexp.cellset1, data.diffexp.cellset2);
const diffExpHistograms = await cxgActions.getAllHistograms(
await subset({ x1: 0.15, y1: 0.1, x2: 0.98, y2: 0.98 });
await runDiffExp(data.diffexp.cellset1, data.diffexp.cellset2);
const diffExpHistograms = await getAllHistograms(
"histogram-diffexp",
diffExpGenes
);
expect(diffExpHistograms).toEqual(expect.arrayContaining(diffExpGenes));
await cxgActions.subset({ x1: 0.16, y1: 0.11, x2: 0.97, y2: 0.97 });
await subset({ x1: 0.16, y1: 0.11, x2: 0.97, y2: 0.97 });
const expected = [].concat(userDefinedGenes, diffExpGenes);
const userDefinedHistogramsAfterSubset = await cxgActions.getAllHistograms(
const userDefinedHistogramsAfterSubset = await getAllHistograms(
"histogram-user-gene",
expected
);
expect(userDefinedHistogramsAfterSubset).toEqual(
expect.arrayContaining(expected)
);
@@ -252,38 +314,51 @@ describe("subset", () => {
describe("scatter plot", () => {
test("scatter plot appears", async () => {
await cxgActions.bulkAddGenes(Object.values(data.scatter.genes));
await utils.clickOn(`plot-x-${data.scatter.genes.x}`);
await utils.clickOn(`plot-y-${data.scatter.genes.y}`);
await utils.waitByID("scatterplot");
await goToPage(appUrlBase);
await bulkAddGenes(Object.values(data.scatter.genes));
await clickOn(`plot-x-${data.scatter.genes.x}`);
await clickOn(`plot-y-${data.scatter.genes.y}`);
await waitByID("scatterplot");
});
});
describe("clipping", () => {
test("clip continuous", async () => {
await cxgActions.clip(data.clip.min, data.clip.max);
await goToPage(appUrlBase);
await clip(data.clip.min, data.clip.max);
const histBrushableAreaId = `histogram-${data.clip.metadata}-plot-brushable-area`;
const coords = await cxgActions.calcDragCoordinates(
const coords = await calcDragCoordinates(
histBrushableAreaId,
data.clip["coordinates-as-percent"]
);
await cxgActions.drag(histBrushableAreaId, coords.start, coords.end);
const cellCount = await cxgActions.cellSet(1);
await drag(histBrushableAreaId, coords.start, coords.end);
const cellCount = await getCellSetCount(1);
expect(cellCount).toBe(data.clip.count);
});
test("clip gene", async () => {
await utils.typeInto("gene-search", data.clip.gene);
await goToPage(appUrlBase);
await typeInto("gene-search", data.clip.gene);
await page.keyboard.press("Enter");
await page.waitForSelector(`[data-testid='histogram-${data.clip.gene}']`);
await cxgActions.clip(data.clip.min, data.clip.max);
await clip(data.clip.min, data.clip.max);
const histBrushableAreaId = `histogram-${data.clip.gene}-plot-brushable-area`;
const coords = await cxgActions.calcDragCoordinates(
const coords = await calcDragCoordinates(
histBrushableAreaId,
data.clip["coordinates-as-percent"]
);
await cxgActions.drag(histBrushableAreaId, coords.start, coords.end);
const cellCount = await cxgActions.cellSet(1);
await drag(histBrushableAreaId, coords.start, coords.end);
const cellCount = await getCellSetCount(1);
expect(cellCount).toBe(data.clip["gene-cell-count"]);
});
});
@@ -291,85 +366,92 @@ describe("clipping", () => {
// interact with UI elements just that they do not break
describe("ui elements don't error", () => {
test("color by", async () => {
for (const label in data.categorical) {
await utils.clickOn(`colorby-${label}`);
}
for (const label in data.continuous) {
await utils.clickOn(`colorby-${label}`);
await goToPage(appUrlBase);
const allLabels = [
...Object.keys(data.categorical),
...Object.keys(data.continuous),
];
for (const label of allLabels) {
await clickOn(`colorby-${label}`);
}
});
test("color by for gene", async () => {
await utils.typeInto("gene-search", data.genes.search);
await goToPage(appUrlBase);
await typeInto("gene-search", data.genes.search);
await page.keyboard.press("Enter");
await page.waitForSelector(
`[data-testid='histogram-${data.genes.search}']`
);
await utils.clickOn(`colorby-${data.genes.search}`);
await clickOn(`colorby-${data.genes.search}`);
});
test("pan and zoom", async () => {
await utils.clickOn("mode-pan-zoom");
const panCoords = await cxgActions.calcDragCoordinates(
await goToPage(appUrlBase);
await clickOn("mode-pan-zoom");
const panCoords = await calcDragCoordinates(
"layout-graph",
data.pan["coordinates-as-percent"]
);
await cxgActions.drag(
"layout-graph",
panCoords.start,
panCoords.end,
false
);
await drag("layout-graph", panCoords.start, panCoords.end, false);
await page.evaluate("window.scrollBy(0, 1000);");
});
});
describe("centroid labels", () => {
test("labels are created", async () => {
await goToPage(appUrlBase);
const labels = Object.keys(data.categorical);
await utils.clickOn(`colorby-${labels[0]}`);
await utils.clickOn("centroid-label-toggle");
/* eslint-disable no-await-in-loop */
await clickOn(`colorby-${labels[0]}`);
await clickOn("centroid-label-toggle");
// Toggle colorby for each category and check to see if labels are generated
for (let i = 0, { length } = labels; i < length; i += 1) {
const label = labels[i];
// first label is already enabled
if (i !== 0) await utils.clickOn(`colorby-${label}`);
const generatedLabels = await utils.getAllByClass("centroid-label");
if (i !== 0) await clickOn(`colorby-${label}`);
const generatedLabels = await getAllByClass("centroid-label");
// Number of labels generated should be equal to size of the object
expect(generatedLabels).toHaveLength(
Object.keys(data.categorical[label]).length
);
}
/* eslint-enable no-await-in-loop */
});
});
describe("graph overlay", () => {
test("transform centroids correctly", async () => {
await goToPage(appUrlBase);
const category = Object.keys(data.categorical)[0];
await utils.clickOn(`colorby-${category}`);
await utils.clickOn("centroid-label-toggle");
await utils.clickOn("mode-pan-zoom");
const panCoords = await cxgActions.calcDragCoordinates(
await clickOn(`colorby-${category}`);
await clickOn("centroid-label-toggle");
await clickOn("mode-pan-zoom");
const panCoords = await calcDragCoordinates(
"layout-graph",
data.pan["coordinates-as-percent"]
);
const categoryValue = Object.keys(data.categorical[category])[0];
const initialCoordinates = await utils.getElementCoordinates(
const initialCoordinates = await getElementCoordinates(
`${categoryValue}-centroid-label`
);
await cxgActions.drag(
"layout-graph",
panCoords.start,
panCoords.end,
false
);
const terminalCoordinates = await utils.getElementCoordinates(
await drag("layout-graph", panCoords.start, panCoords.end, false);
const terminalCoordinates = await getElementCoordinates(
`${categoryValue}-centroid-label`
);
expect(terminalCoordinates[0] - initialCoordinates[0]).toBeCloseTo(
panCoords.end.x - panCoords.start.x
);
@@ -378,3 +460,62 @@ describe("graph overlay", () => {
);
});
});
test("pan zoom mode resets lasso selection", async () => {
await goToPage(appUrlBase);
const panzoomLasso = data.features.panzoom.lasso;
const lassoSelection = await calcDragCoordinates(
"layout-graph",
panzoomLasso["coordinates-as-percent"]
);
await drag("layout-graph", lassoSelection.start, lassoSelection.end, true);
await waitByID("lasso-element", { visible: true });
const initialCount = await getCellSetCount(1);
expect(initialCount).toBe(panzoomLasso.count);
await clickOn("mode-pan-zoom");
await clickOn("mode-lasso");
const modeSwitchCount = await getCellSetCount(1);
expect(modeSwitchCount).toBe(initialCount);
});
test("lasso moves after pan", async () => {
await goToPage(appUrlBase);
const panzoomLasso = data.features.panzoom.lasso;
const coordinatesAsPercent = panzoomLasso["coordinates-as-percent"];
const lassoSelection = await calcDragCoordinates(
"layout-graph",
coordinatesAsPercent
);
await drag("layout-graph", lassoSelection.start, lassoSelection.end, true);
await waitByID("lasso-element", { visible: true });
const initialCount = await getCellSetCount(1);
expect(initialCount).toBe(panzoomLasso.count);
await clickOn("mode-pan-zoom");
const panCoords = await calcDragCoordinates(
"layout-graph",
coordinatesAsPercent
);
await drag("layout-graph", panCoords.start, panCoords.end, false);
await clickOn("mode-lasso");
const panCount = await getCellSetCount(2);
expect(panCount).toBe(initialCount);
});
/* eslint-enable no-await-in-loop -- await in loop is needed to emulate sequential user actions */

View File

@@ -1,208 +1,304 @@
/*
Tests included in this file are specific to annotation features
*/
import { appUrlBase, DATASET, DEBUG } from "./config";
import setupTestBrowser from "./testBrowser";
import { appUrlBase, DATASET } from "./config";
import { datasets } from "./data";
let browser;
let page;
let utils;
let actions;
import {
clickOn,
goToPage,
waitByClass,
waitByID,
getTestId,
getTestClass,
getAllByClass,
} from "./puppeteerUtils";
import {
assertCategoryDoesNotExist,
calcDragCoordinates,
createCategory,
createLabel,
deleteCategory,
deleteLabel,
drag,
expandCategory,
renameCategory,
renameLabel,
subset,
duplicateCategory,
} from "./cellxgeneActions";
const data = datasets[DATASET];
beforeAll(async () => {
[browser, page, utils, actions] = await setupTestBrowser();
});
const perTestCategoryName = "TEST-CATEGORY";
const perTestLabelName = "TEST-LABEL";
afterAll(async () => {
if (!DEBUG && browser !== undefined) await browser.close();
});
async function setup(config) {
await goToPage(appUrlBase);
// setup the test fixtures
await createCategory(perTestCategoryName);
await createLabel(perTestCategoryName, perTestLabelName);
if (config.withSubset) {
await subset({ x1: 0.1, y1: 0.1, x2: 0.8, y2: 0.8 });
}
await waitByClass("autosave-complete");
}
describe.each([
{ withSubset: true, tag: "subset" },
{ withSubset: false, tag: "whole" },
])("annotations", (config) => {
const perTestCategoryName = "per-test-category";
const perTestLabelName = "per-test-label";
beforeEach(async () => {
await page.goto(appUrlBase);
// wait for the page to load
await utils.waitByClass("autosave-complete");
// setup the test fixtures
await actions.createCategory(perTestCategoryName);
await actions.createLabel(perTestCategoryName, perTestLabelName);
if (config.withSubset)
await actions.subset({ x1: 0.1, y1: 0.1, x2: 0.8, y2: 0.8 });
await utils.waitByClass("autosave-complete");
});
afterEach(async () => {
await deleteCategoryIfExists(perTestCategoryName);
await utils.waitByClass("autosave-complete");
});
test("create a category", async () => {
await setup(config);
const categoryName = `category-created-${config.tag}`;
await assertCategoryDoesNotExist(categoryName);
await actions.createCategory(categoryName);
await createCategory(categoryName);
await assertCategoryExists(categoryName);
});
test("delete a category", async () => {
await actions.deleteCategory(perTestCategoryName);
await setup(config);
await deleteCategory(perTestCategoryName);
await assertCategoryDoesNotExist(perTestCategoryName);
});
test("rename a category", async () => {
const newCategoryName = `cluster-for-real-${config.tag}`;
await actions.renameCategory(perTestCategoryName, newCategoryName);
await setup(config);
const newCategoryName = `NEW-${config.tag}`;
await renameCategory(perTestCategoryName, newCategoryName);
await assertCategoryDoesNotExist(perTestCategoryName);
await assertCategoryExists(newCategoryName);
});
test("create a label", async () => {
await setup(config);
const labelName = `new-label-${config.tag}`;
await assertLabelDoesNotExist(perTestCategoryName, labelName);
await actions.createLabel(perTestCategoryName, labelName);
await createLabel(perTestCategoryName, labelName);
await assertLabelExists(perTestCategoryName, labelName);
});
test("delete a label", async () => {
await actions.deleteLabel(perTestCategoryName, perTestLabelName);
await setup(config);
await deleteLabel(perTestCategoryName, perTestLabelName);
await assertLabelDoesNotExist(perTestCategoryName, perTestLabelName);
});
test("rename a label", async () => {
await setup(config);
const newLabelName = "my-cool-new-label";
await assertLabelDoesNotExist(perTestCategoryName, newLabelName);
await actions.renameLabel(
perTestCategoryName,
perTestLabelName,
newLabelName
);
await renameLabel(perTestCategoryName, perTestLabelName, newLabelName);
await assertLabelDoesNotExist(perTestCategoryName, perTestLabelName);
await assertLabelExists(perTestCategoryName, newLabelName);
});
test("check cell count for a label loaded from file", async () => {
const categoryName = "cluster-test";
const labelName = "four";
await actions.expandCategory(categoryName);
const result = await utils.waitByID(
`categorical-value-count-${categoryName}-${labelName}`
await setup(config);
const duplicateCategoryName = "duplicate";
await duplicateCategory(duplicateCategoryName);
await page.reload({ waitUntil: ["networkidle0", "domcontentloaded"] });
const firstCategoryExpandIcon = await expect(page).toMatchElement(
getTestClass("category-expand")
);
expect(await result.evaluate((node) => node.innerText)).toBe(
data.annotationsFromFile.count.bySubsetConfig[config.withSubset]
await firstCategoryExpandIcon.click();
const expectedCategoryRow = await expect(page).toMatchElement(
getTestClass("categorical-row")
);
const expectedLabelName = await getInnerText(
expectedCategoryRow,
"categorical-value"
);
const expectedLabelCount = await getInnerText(
expectedCategoryRow,
"categorical-value-count"
);
await expandCategory(duplicateCategoryName);
const expectedCategory = await expect(page).toMatchElement(
getTestClass("category")
);
const actualCategoryRow = await expect(expectedCategory).toMatchElement(
getTestClass("categorical-row")
);
const actualLabelName = await getInnerText(
actualCategoryRow,
"categorical-value"
);
const actualLabelCount = await getInnerText(
actualCategoryRow,
"categorical-value-count"
);
expect(actualLabelName).toBe(expectedLabelName);
expect(actualLabelCount).toBe(expectedLabelCount);
async function getInnerText(element, className) {
return element.$eval(getTestClass(className), (node) => node?.innerText);
}
});
test("assign cells to a label", async () => {
await actions.expandCategory(perTestCategoryName);
const lassoSelection = await actions.calcDragCoordinates(
await setup(config);
await expandCategory(perTestCategoryName);
const lassoSelection = await calcDragCoordinates(
"layout-graph",
data.categoryLabel.lasso["coordinates-as-percent"]
);
await actions.drag(
"layout-graph",
lassoSelection.start,
lassoSelection.end,
true
);
await utils.waitByID("lasso-element", { visible: true });
await utils.clickOn(
`${perTestCategoryName}:${perTestLabelName}:see-actions`
);
await utils.clickOn(
await drag("layout-graph", lassoSelection.start, lassoSelection.end, true);
await waitByID("lasso-element", { visible: true });
await clickOn(`${perTestCategoryName}:${perTestLabelName}:see-actions`);
await clickOn(
`${perTestCategoryName}:${perTestLabelName}:add-current-selection-to-this-label`
);
const result = await utils.waitByID(
const result = await waitByID(
`categorical-value-count-${perTestCategoryName}-${perTestLabelName}`
);
expect(await result.evaluate((node) => node.innerText)).toBe(
data.categoryLabel.newCount.bySubsetConfig[config.withSubset]
);
});
test("undo/redo category creation", async () => {
await setup(config);
const categoryName = `category-created-undo-${config.tag}`;
await assertCategoryDoesNotExist(categoryName);
await actions.createCategory(categoryName);
await createCategory(categoryName);
await assertCategoryExists(categoryName);
await utils.clickOn("undo");
await clickOn("undo");
await assertCategoryDoesNotExist(categoryName);
await utils.clickOn("redo");
await clickOn("redo");
await assertCategoryExists(categoryName);
});
test("undo/redo category deletion", async () => {
await setup(config);
const categoryName = `category-deleted-undo-${config.tag}`;
await actions.createCategory(categoryName);
await createCategory(categoryName);
await assertCategoryExists(categoryName);
await actions.deleteCategory(categoryName);
await deleteCategory(categoryName);
await assertCategoryDoesNotExist(categoryName);
await utils.clickOn("undo");
await clickOn("undo");
await assertCategoryExists(categoryName);
await utils.clickOn("redo");
await clickOn("redo");
await assertCategoryDoesNotExist(categoryName);
});
test("undo/redo category rename", async () => {
await setup(config);
const newCategoryName = `category-renamed-undo-${config.tag}`;
await assertCategoryDoesNotExist(newCategoryName);
await actions.renameCategory(perTestCategoryName, newCategoryName);
await renameCategory(perTestCategoryName, newCategoryName);
await assertCategoryExists(newCategoryName);
await assertCategoryDoesNotExist(perTestCategoryName);
await utils.clickOn("undo");
await clickOn("undo");
await assertCategoryExists(perTestCategoryName);
await assertCategoryDoesNotExist(newCategoryName);
await utils.clickOn("redo");
await clickOn("redo");
await assertCategoryExists(newCategoryName);
await assertCategoryDoesNotExist(perTestCategoryName);
});
test("undo/redo label creation", async () => {
await setup(config);
const labelName = `label-created-undo-${config.tag}`;
await assertLabelDoesNotExist(perTestCategoryName, labelName);
await actions.createLabel(perTestCategoryName, labelName);
await createLabel(perTestCategoryName, labelName);
await assertLabelExists(perTestCategoryName, labelName);
await utils.clickOn("undo");
await clickOn("undo");
await assertLabelDoesNotExist(perTestCategoryName);
await utils.clickOn("redo");
await clickOn("redo");
await assertLabelExists(perTestCategoryName, labelName);
});
test("undo/redo label deletion", async () => {
await actions.deleteLabel(perTestCategoryName, perTestLabelName);
await setup(config);
await deleteLabel(perTestCategoryName, perTestLabelName);
await assertLabelDoesNotExist(perTestCategoryName);
await utils.clickOn("undo");
await clickOn("undo");
await assertLabelExists(perTestCategoryName, perTestLabelName);
await utils.clickOn("redo");
await clickOn("redo");
await assertLabelDoesNotExist(perTestCategoryName);
});
test("undo/redo label rename", async () => {
await setup(config);
const newLabelName = `label-renamed-undo-${config.tag}`;
await assertLabelDoesNotExist(perTestCategoryName, newLabelName);
await actions.renameLabel(
perTestCategoryName,
perTestLabelName,
newLabelName
);
await renameLabel(perTestCategoryName, perTestLabelName, newLabelName);
await assertLabelExists(perTestCategoryName, newLabelName);
await assertLabelDoesNotExist(perTestCategoryName, perTestLabelName);
await utils.clickOn("undo");
await clickOn("undo");
await assertLabelExists(perTestCategoryName, perTestLabelName);
await assertLabelDoesNotExist(perTestCategoryName, newLabelName);
await utils.clickOn("redo");
await clickOn("redo");
await assertLabelExists(perTestCategoryName, newLabelName);
await assertLabelDoesNotExist(perTestCategoryName, perTestLabelName);
});
test("stacked bar graph renders", async () => {
await setup(config);
await expandCategory(perTestCategoryName);
await clickOn(`colorby-louvain`);
const labels = await getAllByClass("categorical-row");
const result = await Promise.all(
labels.map((label) => {
return page.evaluate((element) => {
return element.outerHTML;
}, label);
})
);
expect(result).toMatchSnapshot();
});
async function assertCategoryExists(categoryName) {
const handle = await utils.waitByID(`${categoryName}:category-label`);
const handle = await waitByID(`${categoryName}:category-label`);
const result = await handle.evaluate((node) =>
node.getAttribute("aria-label")
@@ -211,37 +307,27 @@ describe.each([
expect(result).toBe(categoryName);
}
async function assertCategoryDoesNotExist(categoryName) {
const result = await page.$(
`[data-testid='${categoryName}:category-label']`
);
expect(result).toBeNull();
}
async function assertLabelExists(categoryName, labelName) {
const category = await utils.waitByID(`${categoryName}:category-expand`);
expect(category).not.toBeNull();
await actions.expandCategory(categoryName);
const previous = await utils.waitByID(
await expect(page).toMatchElement(
getTestId(`${categoryName}:category-expand`)
);
await expandCategory(categoryName);
const previous = await waitByID(
`categorical-value-${categoryName}-${labelName}`
);
expect(
await previous.evaluate((node) => node.getAttribute("aria-label"))
).toBe(labelName);
}
async function assertLabelDoesNotExist(categoryName, labelName) {
await actions.expandCategory(categoryName);
await expandCategory(categoryName);
const result = await page.$(
`[data-testid='categorical-value-${categoryName}-${labelName}']`
);
expect(result).toBeNull();
}
async function deleteCategoryIfExists(categoryName) {
const handle = await page.$(
`[data-testid='${categoryName}:category-expand']`
);
if (handle) await actions.deleteCategory(categoryName);
}
});

View File

@@ -1,5 +1,10 @@
{
"testRunner": "jest-circus/runner",
"preset": "jest-puppeteer",
"testMatch": ["**/__tests__/**/?(*.)(spec|test).js?(x)"],
"setupFiles": ["../setupMissingGlobals.js"]
"setupFiles": ["../setupMissingGlobals.js"],
"setupFilesAfterEnv": ["expect-puppeteer", "./puppeteer.setup.js"],
"globalSetup": "jest-environment-puppeteer/setup",
"globalTeardown": "jest-environment-puppeteer/teardown",
"testEnvironment": "./screenshot_env.js"
}

View File

@@ -1,122 +0,0 @@
/*
NOT run in Travis CI
UX tests using puppeteer to be run locally.
To run locally, ensure you are running the client is running on port 3000.
Then run jest --verbose false --config __tests__/e2e/e2eJestConfig.json feature.
*/
import puppeteer from "puppeteer";
import { appUrlBase, DEBUG, DEV, DATASET } from "./config";
import { puppeteerUtils, cellxgeneActions } from "./puppeteerUtils";
import { datasets } from "./data";
let browser;
let page;
let utils;
let cxgActions;
let spy;
const browserViewport = { width: 1280, height: 960 };
const data = datasets[DATASET].features;
if (DEBUG) jest.setTimeout(100000);
if (DEV) jest.setTimeout(10000);
beforeAll(async () => {
const browserParams = DEV
? { headless: false, slowMo: 5 }
: DEBUG
? { headless: false, slowMo: 100, devtools: true }
: {};
browser = await puppeteer.launch(browserParams);
page = await browser.newPage();
await page.setViewport(browserViewport);
if (DEV || DEBUG) {
page.on("console", (msg) => console.log(`PAGE LOG: ${msg.text()}`));
}
page.on("pageerror", (err) => {
throw new Error(`Console error: ${err}`);
});
utils = puppeteerUtils(page);
cxgActions = cellxgeneActions(page);
});
beforeEach(async () => {
await page.goto(appUrlBase);
});
afterAll(() => {
if (!DEBUG) {
browser.close();
}
});
describe("zoom interaction", async () => {
// Skip this test since UI is to hide lasso path when switching modes
test.skip("lasso visible after switching modes to pan/zoom", async () => {
const lassoSelection = await cxgActions.calcDragCoordinates(
"layout-graph",
data.panzoom.lasso["coordinates-as-percent"]
);
await cxgActions.drag(
"layout-graph",
lassoSelection.start,
lassoSelection.end,
true
);
await utils.waitByID("lasso-element", { visible: true });
await utils.clickOn("mode-pan-zoom");
await utils.waitByID("lasso-element", { visible: true });
});
test("pan zoom mode resets lasso selection", async () => {
const lassoSelection = await cxgActions.calcDragCoordinates(
"layout-graph",
data.panzoom.lasso["coordinates-as-percent"]
);
await cxgActions.drag(
"layout-graph",
lassoSelection.start,
lassoSelection.end,
true
);
await utils.waitByID("lasso-element", { visible: true });
const initialCount = await cxgActions.cellSet(1);
expect(initialCount).toBe(data.panzoom.lasso.count);
await utils.clickOn("mode-pan-zoom");
await utils.clickOn("mode-lasso");
const modeSwitchCount = await cxgActions.cellSet(1);
expect(modeSwitchCount).toBe(initialCount);
});
test("lasso moves after pan", async () => {
const lassoSelection = await cxgActions.calcDragCoordinates(
"layout-graph",
data.panzoom.lasso["coordinates-as-percent"]
);
await cxgActions.drag(
"layout-graph",
lassoSelection.start,
lassoSelection.end,
true
);
await utils.waitByID("lasso-element", { visible: true });
const initialCount = await cxgActions.cellSet(1);
expect(initialCount).toBe(data.panzoom.lasso.count);
await utils.clickOn("mode-pan-zoom");
const panCoords = await cxgActions.calcDragCoordinates(
"layout-graph",
data.panzoom.lasso["coordinates-as-percent"]
);
await cxgActions.drag(
"layout-graph",
panCoords.start,
panCoords.end,
false
);
await utils.clickOn("mode-lasso");
const panCount = await cxgActions.cellSet(2);
expect(panCount).toBe(initialCount);
});
});

View File

@@ -0,0 +1,60 @@
/**
* `client/jest-puppeteer.config.js` is for configuring Puppeteer's launch config options
* `client/__tests__/e2e/puppeteer.setup.js` is for configuring `jest`, `browser`,
* and `page` objects
*/
import { setDefaultOptions } from "expect-puppeteer";
import { isDebug, isDev } from "./config";
import * as ENV_DEFAULT from "../../../environment.default.json";
// (thuang): This is the max time a test can take to run.
// Since when debugging, we run slowMo and !headless, this means
// a test can take more time to finish, so we don't want
// jest to shut off the test too soon
jest.setTimeout(2 * 60 * 1000);
setDefaultOptions({ timeout: 20 * 1000 });
jest.retryTimes(ENV_DEFAULT.RETRY_ATTEMPTS);
beforeEach(async () => {
await jestPuppeteer.resetBrowser();
const userAgent = await browser.userAgent();
await page.setUserAgent(`${userAgent}bot`);
await page._client.send("Animation.setPlaybackRate", { playbackRate: 12 });
page.on("console", async (msg) => {
page.on("pageerror", (err) => {
console.log(`PAGE LOG: ${msg.text()}`);
throw new Error(`Console error: ${err}`);
});
page.on("error", (err) => {
console.log(`PAGE LOG: ${msg.text()}`);
throw new Error(`Console error: ${err}`);
});
if (isDev || isDebug) {
// If there is a console.error but an error is not thrown, this will ensure the test fails
console.log(`PAGE LOG: ${msg.text()}`);
if (msg.type() === "error") {
// TODO: chromium does not currently support the CSP directive on the
// line below, so we swallow this error. Remove this when the test
// suite uses a browser version that supports this directive.
if (
msg.text() ===
"Unrecognized Content-Security-Policy directive 'require-trusted-types-for'.\n"
) {
return;
}
const errorMsgText = await Promise.all(
// TODO can we do this without internal properties?
msg.args().map((arg) => arg._remoteObject.description)
);
throw new Error(`Console error: ${errorMsgText}`);
}
}
});
});

View File

@@ -1,73 +1,131 @@
const puppeteerUtils = (page) => ({
waitByID(testId, props = {}) {
return page.waitForSelector(`[data-testid='${testId}']`, props);
},
/* eslint-disable no-await-in-loop -- await in loop is needed to emulate sequential user actions */
export function getTestId(id) {
return `[data-testid='${id}']`;
}
waitByClass(testClass, props = {}) {
return page.waitForSelector(`[data-testclass='${testClass}']`, props);
},
export function getTestClass(className) {
return `[data-testclass='${className}']`;
}
async waitForAllByIds(testIds, props = {}) {
return Promise.all(
testIds.map((testId) =>
page.waitForSelector(`[data-testid='${testId}']`, props)
)
);
},
export async function waitByID(testId, props = {}) {
return page.waitForSelector(getTestId(testId), props);
}
async getAllByClass(testClass) {
return page.$$eval(`[data-testclass=${testClass}]`, (eles) =>
eles.map((ele) => ele.dataset.testid)
);
},
export async function waitByClass(testClass, props = {}) {
await page.waitForSelector(`[data-testclass='${testClass}']`, props);
}
async typeInto(testId, text) {
// blueprint's typeahead is treating typing weird, clicking & waiting first solves this
// only works for text without special characters
await this.waitByID(testId);
const selector = `[data-testid='${testId}']`;
// type ahead can be annoying if you don't pause before you type
await page.click(selector);
await page.waitFor(200);
await page.type(selector, text);
},
export async function waitForAllByIds(testIds) {
await Promise.all(
testIds.map((testId) => page.waitForSelector(getTestId(testId)))
);
}
async clearInputAndTypeInto(testId, text) {
await this.waitByID(testId);
const selector = `[data-testid='${testId}']`;
// only works for text without special characters
// type ahead can be annoying if you don't pause before you type
await page.click(selector);
await page.waitFor(200);
// select all
await page.click(selector, { clickCount: 3 });
await page.keyboard.press("Backspace");
await page.type(selector, text);
},
export async function getAllByClass(testClass) {
return page.$$(`[data-testclass=${testClass}]`);
}
async clickOn(testid, options = {}) {
await this.waitByID(testid, options);
const click = await page.click(`[data-testid='${testid}']`);
await page.waitFor(50);
return click;
},
export async function typeInto(testId, text) {
// blueprint's typeahead is treating typing weird, clicking & waiting first solves this
// only works for text without special characters
await waitByID(testId);
const selector = getTestId(testId);
// type ahead can be annoying if you don't pause before you type
await page.click(selector);
await page.waitFor(200);
await page.type(selector, text);
}
async getOneElementInnerHTML(selector, options = {}) {
await page.waitForSelector(selector, options);
return page.$eval(selector, (el) => el.innerHTML);
},
export async function clearInputAndTypeInto(testId, text) {
await waitByID(testId);
const selector = getTestId(testId);
// only works for text without special characters
// type ahead can be annoying if you don't pause before you type
await page.click(selector);
await page.waitFor(200);
// select all
await page.click(selector, { clickCount: 3 });
await page.keyboard.press("Backspace");
await page.type(selector, text);
}
async getOneElementInnerText(selector) {
await page.waitForSelector(selector);
return page.$eval(selector, (el) => el.innerText);
},
export async function clickOn(testId, options = {}) {
await expect(page).toClick(getTestId(testId), options);
}
async getElementCoordinates(testid) {
return page.$eval(`[data-testid='${testid}']`, (elem) => {
const { left, top } = elem.getBoundingClientRect();
return [left, top];
});
},
});
/**
* (thuang): There are times when Puppeteer clicks on a button and the page doesn't respond.
* So I added clickOnUntil() to retry clicking until a given condition is met.
*/
export async function clickOnUntil(testId, assert) {
const MAX_RETRY = 10;
const WAIT_FOR_MS = 200;
export default puppeteerUtils;
let retry = 0;
while (retry < MAX_RETRY) {
try {
await clickOn(testId);
await assert();
break;
} catch (error) {
retry += 1;
await page.waitFor(WAIT_FOR_MS);
}
}
if (retry === MAX_RETRY) {
throw Error("clickOnUntil() assertion failed!");
}
}
export async function getOneElementInnerHTML(selector, options = {}) {
await page.waitForSelector(selector, options);
return page.$eval(selector, (el) => el.innerHTML);
}
export async function getOneElementInnerText(selector) {
expect(page).toMatchElement(selector);
return page.$eval(selector, (el) => el.innerText);
}
export async function getElementCoordinates(testId) {
return page.$eval(getTestId(testId), (elem) => {
const { left, top } = elem.getBoundingClientRect();
return [left, top];
});
}
async function clickTermsOfService() {
if (!(await isElementPresent(getTestId("tos-cookies-accept")))) return;
await clickOn("tos-cookies-accept");
}
async function nameNewAnnotation() {
if (await isElementPresent(getTestId("annotation-dialog"))) {
await typeInto("new-annotation-name", "ignoreE2E");
await clickOn("submit-annotation");
// wait for the page to load
await waitByClass("autosave-complete");
}
}
export async function goToPage(url) {
await page.goto(url, {
waitUntil: "networkidle0",
});
await nameNewAnnotation();
await clickTermsOfService();
}
export async function isElementPresent(selector, options) {
return Boolean(await page.$(selector, options));
}
/* eslint-enable no-await-in-loop -- await in loop is needed to emulate sequential user actions */

View File

@@ -0,0 +1,28 @@
const PuppeteerEnvironment = require("jest-environment-puppeteer");
require("jest-circus");
const ENV_DEFAULT = require("../../../environment.default.json");
const takeScreenshot = require("./takeScreenshot");
class ScreenshotEnvironment extends PuppeteerEnvironment {
async handleTestEvent(event, state) {
if (event.name === "error") {
console.log("error", JSON.stringify(event));
}
if (event.name === "test_fn_failure" || event.name === "hook_failure") {
// (thuang): We only want to take screenshot on the last try
if (
state.currentlyRunningTest.invocations <= ENV_DEFAULT.RETRY_ATTEMPTS
) {
return;
}
console.log("===> Failure event\n", new Date(), event);
await takeScreenshot(state.currentlyRunningTest.name, this.global.page);
}
}
}
module.exports = ScreenshotEnvironment;

View File

@@ -0,0 +1,17 @@
function toFilename(name) {
return name.replace(/[^a-z0-9.-]+/gi, "-");
}
async function takeScreenshot(currentTestName, page) {
const testName = toFilename(currentTestName);
// Take a screenshot at the point of failure
const date = new Date().toISOString();
const screenshotName = `${date}-${testName}.png`;
await page.screenshot({
path: `./__tests__/screenshots/ignoreE2E-screenshot-${screenshotName}`,
});
}
module.exports = takeScreenshot;

View File

@@ -1,67 +0,0 @@
import puppeteer from "puppeteer";
import { DEBUG, DEV } from "./config";
import puppeteerUtils from "./puppeteerUtils";
import cellxgeneActions from "./cellxgeneActions";
export default async function setupTestBrowser() {
const browserViewport = { width: 1280, height: 960 };
const browserParams = DEV
? {
headless: false,
slowMo: 5,
args: [
`--window-size=${browserViewport.width},${browserViewport.height}`,
],
}
: DEBUG
? {
headless: false,
slowMo: 100,
devtools: true,
args: [
`--window-size=${browserViewport.width + 560},${
browserViewport.height
}`,
],
}
: {
args: [
`--window-size=${browserViewport.width},${browserViewport.height}`,
],
};
const browser = await puppeteer.launch(browserParams);
const page = await browser.pages().then((pages) => pages[0]);
await page.setViewport(browserViewport);
if (DEV || DEBUG) {
page.on("console", async (msg) => {
// If there is a console.error but an error is not thrown, this will ensure the test fails
console.log(`PAGE LOG: ${msg.text()}`);
if (msg.type() === "error") {
// TODO: chromium does not currently support the CSP directive on the
// line below, so we swallow this error. Remove this when the test
// suite uses a browser version that supports this directive.
if (
msg.text() ===
"Unrecognized Content-Security-Policy directive 'require-trusted-types-for'.\n"
)
return;
const errorMsgText = await Promise.all(
// TODO can we do this without internal properties?
msg.args().map((arg) => arg._remoteObject.description)
);
throw new Error(`Console error: ${errorMsgText}`);
}
});
}
page.on("pageerror", (err) => {
console.log(`PAGE LOG: ${msg.text()}`);
throw new Error(`Console error: ${err}`);
});
page.on("error", (err) => {
console.log(`PAGE LOG: ${msg.text()}`);
throw new Error(`Console error: ${err}`);
});
const utils = puppeteerUtils(page);
const cxgActions = cellxgeneActions(page, utils);
return [browser, page, utils, cxgActions];
}

View File

@@ -0,0 +1,4 @@
# Ignore everything in this directory
*
# Except this file
!.gitignore

View File

@@ -7,11 +7,10 @@ import * as Universe from "../../src/util/stateManager/universe";
import { matrixFBSToDataframe } from "../../src/util/stateManager/matrix";
import * as World from "../../src/util/stateManager/world";
import * as REST from "./stateManager/sampleResponses";
import { ControlsHelpers as CH } from "../../src/util/stateManager";
describe("centroid", () => {
let world;
let categoricalSelection;
beforeAll(() => {
// Create world + universe
let universe = Universe.createUniverseFromResponse(
@@ -32,19 +31,10 @@ describe("centroid", () => {
...Universe.addObsLayout(universe, matrixFBSToDataframe(REST.layoutObs)),
};
world = World.createWorldFromEntireUniverse(universe);
// Create categorical selection from world
categoricalSelection = CH.createCategoricalSelection(
CH.selectableCategoryNames(world.schema)
);
});
test("field4 (categorical obsAnnotation)", () => {
const centroidResult = calcCentroid(
world,
"field4",
["umap_0", "umap_1"]
);
const centroidResult = calcCentroid(world, "field4", ["umap_0", "umap_1"]);
// Check to see that a centroid has been calculated for every categorical value
const keysAsArray = Array.from(centroidResult.keys());
@@ -64,11 +54,7 @@ describe("centroid", () => {
});
test("field3 (boolean obsAnnotation)", () => {
const centroidResult = calcCentroid(
world,
"field3",
["umap_0", "umap_1"]
);
const centroidResult = calcCentroid(world, "field3", ["umap_0", "umap_1"]);
// Check to see that a centroid has been calculated for every categorical value
const keysAsArray = Array.from(centroidResult.keys());

View File

@@ -918,43 +918,59 @@ describe("dataframe col", () => {
});
describe("label indexing", () => {
test("IdentityInt32Index", () => {
const idx = new Dataframe.IdentityInt32Index(12); // [0, 12)
expect(Dataframe.isLabelIndex(idx)).toBeTruthy();
expect(idx.labels()).toEqual(new Int32Array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]));
expect(idx.labels()).toEqual(
new Int32Array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
);
expect(idx.getLabel(1)).toEqual(1);
expect(idx.getOffset(1)).toEqual(1);
expect(idx.getOffsets([1,3])).toEqual([1,3])
expect(idx.getLabels([1, 3])).toEqual([1,3])
expect(idx.getOffsets([1, 3])).toEqual([1, 3]);
expect(idx.getLabels([1, 3])).toEqual([1, 3]);
expect(idx.size()).toEqual(12);
expect(idx.subset([2]).labels()).toEqual([2]);
expect(idx.subset([2, 3, 4]).labels()).toEqual(new Int32Array([2, 3, 4]));
expect(idx.subset([0, 1, 2, 3]).labels()).toEqual(new Int32Array([0, 1, 2, 3]));
expect(idx.subset([0, 1, 2, 3]).labels()).toEqual(
new Int32Array([0, 1, 2, 3])
);
expect(idx.isubset([2]).labels()).toEqual([2]);
expect(idx.isubset([2, 3, 4]).labels()).toEqual(new Int32Array([2, 3, 4]));
expect(idx.isubset([0, 1, 2, 3]).labels()).toEqual(new Int32Array([0, 1, 2, 3]));
expect(idx.isubset([0, 1, 2, 3]).labels()).toEqual(
new Int32Array([0, 1, 2, 3])
);
expect(idx.subset([0, 1, 2, 3, 4])).toBeInstanceOf(Dataframe.IdentityInt32Index);
expect(idx.subset([0, 1, 2, 3, 4])).toBeInstanceOf(
Dataframe.IdentityInt32Index
);
expect(idx.subset([2, 1, 0])).toBeInstanceOf(Dataframe.IdentityInt32Index);
expect(idx.subset([1, 2, 3, 4])).toBeInstanceOf(Dataframe.DenseInt32Index);
expect(idx.subset([0, 1, 3, 4])).toBeInstanceOf(Dataframe.DenseInt32Index);
expect(idx.subset([0, 1, 2, 3, 10])).toBeInstanceOf(Dataframe.DenseInt32Index);
expect(idx.subset([0, 1, 2, 3, 10])).toBeInstanceOf(
Dataframe.DenseInt32Index
);
expect(idx.subset([4, 3, 2, 1])).toBeInstanceOf(Dataframe.DenseInt32Index);
expect(idx.subset([4])).toBeInstanceOf(Dataframe.KeyIndex);
expect(idx.withLabel(99).labels()).toEqual(new Int32Array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 99]));
expect(idx.dropLabel(0).labels()).toEqual(new Int32Array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]));
expect(idx.dropLabel(11).labels()).toEqual(new Int32Array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]));
expect(idx.dropLabel(5).labels()).toEqual(new Int32Array([0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11]));
expect(idx.withLabel(99).labels()).toEqual(
new Int32Array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 99])
);
expect(idx.dropLabel(0).labels()).toEqual(
new Int32Array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
);
expect(idx.dropLabel(11).labels()).toEqual(
new Int32Array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
);
expect(idx.dropLabel(5).labels()).toEqual(
new Int32Array([0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11])
);
});
test("DenseInt32Index", () => {
const idx = new Dataframe.DenseInt32Index([99, 1002, 48, 0, 22]);
expect(Dataframe.isLabelIndex(idx)).toBeTruthy();
@@ -964,17 +980,29 @@ describe("label indexing", () => {
expect(idx.getOffset(1002)).toEqual(1);
expect(idx.getOffset(0)).toEqual(3);
expect(idx.getLabel(0)).toEqual(99);
expect(idx.getLabels(new Int32Array([2, 4]))).toEqual(new Int32Array([48, 22]));
expect(idx.getLabels(new Int32Array([2, 4]))).toEqual(
new Int32Array([48, 22])
);
expect(idx.getLabels([2, 4])).toEqual([48, 22]);
expect(idx.getOffsets([0, 48])).toEqual([3, 2]);
expect(idx.subset([1002, 0, 99]).labels()).toEqual(new Int32Array([1002, 0, 99]))
expect(idx.getOffsets(idx.subset([1002, 0, 99]).labels())).toEqual(new Int32Array([1, 3, 0]));
expect(idx.isubset([4, 1, 2]).labels()).toEqual(new Int32Array([22, 1002, 48]));
expect(idx.subset([1002, 0, 99]).labels()).toEqual(
new Int32Array([1002, 0, 99])
);
expect(idx.getOffsets(idx.subset([1002, 0, 99]).labels())).toEqual(
new Int32Array([1, 3, 0])
);
expect(idx.isubset([4, 1, 2]).labels()).toEqual(
new Int32Array([22, 1002, 48])
);
expect(idx.withLabel(88).labels()).toEqual(new Int32Array([99, 1002, 48, 0, 22, 88]));
expect(idx.withLabel(88).labels()).toEqual(
new Int32Array([99, 1002, 48, 0, 22, 88])
);
expect(idx.withLabel(88).getOffset(88)).toEqual(5);
expect(idx.dropLabel(48).labels()).toEqual(new Int32Array([99, 1002, 0, 22]));
expect(idx.dropLabel(48).labels()).toEqual(
new Int32Array([99, 1002, 0, 22])
);
});
test("KeyIndex", () => {
@@ -991,9 +1019,13 @@ describe("label indexing", () => {
expect(idx.subset(["green", "red"]).labels()).toEqual(["green", "red"]);
expect(idx.isubset([2, 1, 0]).labels()).toEqual(["blue", "green", "red"]);
expect(idx.withLabel("yo").labels()).toEqual(["red", "green", "blue", "yo"]);
expect(idx.withLabel("yo").labels()).toEqual([
"red",
"green",
"blue",
"yo",
]);
expect(idx.withLabel("yo").getOffset("yo")).toEqual(3);
expect(idx.dropLabel("blue").labels()).toEqual(["red", "green"]);
});
});
})

View File

@@ -1,7 +1,7 @@
import PromiseLimit from "../../src/util/promiseLimit";
import { range } from "../../src/util/range";
const delay = (t) => new Promise((resolve, reject) => setTimeout(resolve, t));
const delay = (t) => new Promise((resolve) => setTimeout(resolve, t));
describe("PromiseLimit", () => {
test("simple evaluation, concurrency 1", async () => {
@@ -28,13 +28,14 @@ describe("PromiseLimit", () => {
test("eval in order of insertion", async () => {
const plimit = new PromiseLimit(100);
let counter = 0;
const result = await Promise.all([
plimit.add(() => Promise.resolve((counter += 1))),
plimit.add(() => Promise.resolve((counter += 1))),
plimit.add(() => Promise.resolve((counter += 1))),
plimit.add(() => Promise.resolve((counter += 1))),
plimit.add(() => Promise.resolve(1)),
plimit.add(() => Promise.resolve(2)),
plimit.add(() => Promise.resolve(3)),
plimit.add(() => Promise.resolve(4)),
]);
expect(result).toEqual([1, 2, 3, 4]);
});
@@ -43,16 +44,15 @@ describe("PromiseLimit", () => {
let running = 0;
let maxRunning = 0;
const cbfn = async (i) => {
running = running + 1;
const callback = async () => {
running += 1;
maxRunning = running > maxRunning ? running : maxRunning;
await delay(100);
running = running - 1;
running -= 1;
};
const result = await Promise.all(
range(10).map((i) => plimit.add(() => cbfn(i)))
);
await Promise.all(range(10).map((i) => plimit.add(() => callback(i))));
expect(maxRunning).toEqual(2);
});
@@ -60,6 +60,7 @@ describe("PromiseLimit", () => {
const plimit = new PromiseLimit(2);
const result = await Promise.all([
plimit.add(() => Promise.resolve("OK")),
// eslint-disable-next-line prefer-promise-reject-errors -- unit test
plimit.add(() => Promise.reject("not OK")).catch((e) => e),
plimit.add(() => Promise.resolve("OK")),
plimit

View File

@@ -4,7 +4,6 @@ test controls helpers
import { subsetAndResetGeneLists } from "../../../src/util/stateManager/controlsHelpers";
import * as globals from "../../../src/globals";
describe("controls helpers", () => {
test("subsetAndResetGeneLists", () => {
const geneList = [];
@@ -28,7 +27,7 @@ describe("controls helpers", () => {
);
const expectedNewUserDefinedGenes = [
...geneList.slice(0, 20),
...geneList.slice(21)
...geneList.slice(21),
].slice(0, globals.maxGenes);
expect(globals.maxUserDefinedGenes).toBeLessThan(globals.maxGenes);
expect(geneList.length).toBeGreaterThan(globals.maxGenes);

View File

@@ -1,4 +1,3 @@
/* eslint no-bitwise: "off" */
import _ from "lodash";
import { flatbuffers } from "flatbuffers";
import { NetEncoding } from "../../../src/util/stateManager/matrix_generated";
@@ -79,6 +78,7 @@ const anAnnotationsObsJSONResponse = {
`obs${idx}`,
2 * idx,
idx + 0.0133,
// eslint-disable-next-line no-bitwise -- idx & 1 to check for odd numbers
!!(idx & 1),
field4Categories[idx % field4Categories.length],
])
@@ -93,6 +93,7 @@ const anAnnotationsVarJSONResponse = {
idx,
10 * idx,
idx + 2.90143,
// eslint-disable-next-line no-bitwise -- idx & 1 to check for odd numbers
!!(idx & 1),
fieldDCategories[idx % fieldDCategories.length],
`var${idx}`,
@@ -112,7 +113,7 @@ function encodeTypedArray(builder, uType, uData) {
function encodeMatrix(columns, colIndex = undefined) {
/*
IMPORTANT: this is not a general purpose encoder. in particular,
it doesn't correctly handle all column index types, nor does it
it doesn't correctly handle all column index types, nor does it
handle all column typedarray types.
encodeMatrixFBS in matrix.py is more general. This is used only

View File

@@ -2,9 +2,6 @@ import {
sortArray,
sortIndex,
lowerBound,
upperBound,
lowerBoundIndirect,
upperBoundIndirect,
} from "../../../src/util/typedCrossfilter/sort";
/*
@@ -68,7 +65,7 @@ describe("sortArray", () => {
});
describe("non-finite numbers", () => {
test("inifinity", () => {
test("infinity", () => {
expect(sortArray(new Float32Array([pInf, nInf, 0, 1, 2]))).toMatchObject(
new Float32Array([nInf, 0, 1, 2, pInf])
);

View File

@@ -8,7 +8,16 @@ module.exports = {
"prettier/react",
],
env: { browser: true, commonjs: true, es6: true },
globals: { expect: true },
globals: {
expect: true,
jest: true,
jestPuppeteer: true,
it: true,
page: true,
browser: true,
context: true,
beforeEach: true,
},
parserOptions: {
ecmaVersion: 2017,
sourceType: "module",
@@ -36,6 +45,13 @@ module.exports = {
"space-before-function-paren": "off",
"function-paren-newline": "off",
"prefer-destructuring": ["error", { object: true, array: false }],
"import/prefer-default-export": "off",
"no-restricted-syntax": [
"error",
"ForInStatement",
"LabeledStatement",
"WithStatement",
],
},
overrides: [
{

View File

@@ -1,3 +1,3 @@
module.exports = {
"./src/**/*.js": "eslint --fix",
"*.js": "eslint --fix",
};

View File

@@ -0,0 +1,47 @@
/**
* `client/jest-puppeteer.config.js` is for configuring Puppeteer's launch config options
* `client/__tests__/e2e/puppeteer.setup.js` is for configuring `jest`, `browser`,
* and `page` objects
*/
const ENV_DEFAULT = require("../environment.default.json");
const jestEnv = process.env.JEST_ENV || ENV_DEFAULT.JEST_ENV;
const isHeadful =
process.env.HEADFUL === "true" || process.env.HEADLESS === "false";
const DEFAULT_LAUNCH_CONFIG = {
headless: !isHeadful,
args: ["--ignore-certificate-errors", "--ignore-ssl-errors"],
dumpio: true,
ignoreHTTPSErrors: true,
defaultViewport: {
width: 1280,
height: 960,
},
};
const LAUNCH_CONFIG_BY_ENV = {
[ENV_DEFAULT.DEBUG]: {
...DEFAULT_LAUNCH_CONFIG,
headless: false,
slowMo: 100,
devtools: true,
defaultViewport: {
width: DEFAULT_LAUNCH_CONFIG.defaultViewport.width,
height: DEFAULT_LAUNCH_CONFIG.defaultViewport.height + 560,
},
},
[ENV_DEFAULT.DEV]: {
...DEFAULT_LAUNCH_CONFIG,
headless: false,
slowMo: 5,
},
};
const launchConfig = LAUNCH_CONFIG_BY_ENV[jestEnv] || DEFAULT_LAUNCH_CONFIG;
module.exports = {
browserContext: "incognito",
launch: launchConfig,
};

View File

@@ -12541,6 +12541,32 @@
}
}
},
"jest-circus": {
"version": "26.0.1",
"resolved": "https://registry.npmjs.org/jest-circus/-/jest-circus-26.0.1.tgz",
"integrity": "sha512-dp20V0Pi1N92Y7+ULPa3tNR9KCG0Sy19NiopyPmo5rNoQ4OGWmuzp1P0q1je2HV3fRD0BYE7wqh8aReGGENfUA==",
"dev": true,
"requires": {
"@babel/traverse": "^7.1.0",
"@jest/environment": "^26.0.1",
"@jest/test-result": "^26.0.1",
"@jest/types": "^26.0.1",
"chalk": "^4.0.0",
"co": "^4.6.0",
"dedent": "^0.7.0",
"expect": "^26.0.1",
"is-generator-fn": "^2.0.0",
"jest-each": "^26.0.1",
"jest-matcher-utils": "^26.0.1",
"jest-message-util": "^26.0.1",
"jest-runtime": "^26.0.1",
"jest-snapshot": "^26.0.1",
"jest-util": "^26.0.1",
"pretty-format": "^26.0.1",
"stack-utils": "^2.0.2",
"throat": "^5.0.0"
}
},
"jest-config": {
"version": "26.0.1",
"resolved": "https://registry.npmjs.org/jest-config/-/jest-config-26.0.1.tgz",

View File

@@ -5,12 +5,16 @@
"description": "cellxgene is a web application for the interactive exploration of single cell sequence data.",
"repository": "https://github.com/chanzuckerberg/cellxgene",
"scripts": {
"clean": "rimraf build",
"build": "npm run clean && webpack --config",
"prod": "npm run build -- configuration/webpack/webpack.config.prod.js",
"clean": "rimraf build",
"dev": "npm run build -- configuration/webpack/webpack.config.dev.js",
"fmt": "eslint --fix src",
"lint": "eslint src"
"e2e": "jest --config __tests__/e2e/e2eJestConfig.json e2e/e2e.test.js",
"e2e-annotations": "jest --config __tests__/e2e/e2eJestConfig.json e2e/e2eAnnotations.test.js",
"e2e-prod": "CXG_URL_BASE='https://cellxgene.cziscience.com/d/pbmc3k.cxg/' jest --config __tests__/e2e/e2eJestConfig.json e2e/e2e.test.js",
"fmt": "eslint --fix src __tests__",
"lint": "eslint --fix src __tests__",
"prod": "npm run build -- configuration/webpack/webpack.config.prod.js",
"test": "jest --testPathIgnorePatterns e2e"
},
"engineStrict": true,
"engines": {
@@ -88,6 +92,7 @@
"eslint-plugin-prettier": "^3.1.3",
"eslint-plugin-react": "^7.20.0",
"eslint-plugin-react-hooks": "^4.0.4",
"expect-puppeteer": "^4.4.0",
"express": "^4.17.1",
"favicons-webpack-plugin": "^3.0.1",
"file-loader": "^6.0.0",
@@ -95,6 +100,8 @@
"html-webpack-plugin": "^4.3.0",
"husky": "^4.2.5",
"jest": "^26.0.1",
"jest-circus": "^26.0.1",
"jest-environment-puppeteer": "^4.4.0",
"jest-puppeteer": "^4.4.0",
"json-loader": "^0.5.7",
"lint-staged": "^10.2.9",

View File

@@ -113,7 +113,7 @@ class FilenameDialog extends React.Component {
this.handleCreateFilename();
}}
>
<div className={Classes.DIALOG_BODY}>
<div className={Classes.DIALOG_BODY} data-testid="annotation-dialog">
<div style={{ marginBottom: 20 }}>
<p>Name your annotations collection:</p>
<InputGroup
@@ -124,6 +124,7 @@ class FilenameDialog extends React.Component {
this.setState({ filenameText: e.target.value })
}
leftIcon="tag"
data-testid="new-annotation-name"
/>
<p
style={{
@@ -159,6 +160,7 @@ class FilenameDialog extends React.Component {
onClick={this.handleCreateFilename}
intent="primary"
type="submit"
data-testid="submit-annotation"
>
Create annotations collection
</Button>

View File

@@ -27,7 +27,14 @@ class DuplicateCategorySelect extends React.PureComponent {
}
filterable={false}
itemRenderer={(d, { handleClick }) => {
return <MenuItem onClick={handleClick} key={d} text={d} />;
return (
<MenuItem
data-testclass="duplicate-category-dropdown-option"
onClick={handleClick}
key={d}
text={d}
/>
);
}}
noResults={<MenuItem disabled text="No results." />}
onItemSelect={(d) => {
@@ -36,6 +43,7 @@ class DuplicateCategorySelect extends React.PureComponent {
>
{/* children become the popover target; render value here */}
<Button
data-testid="duplicate-category-dropdown"
text={categoryToDuplicate || "None (all cells 'unassigned')"}
rightIcon="double-caret-vertical"
/>

View File

@@ -258,6 +258,7 @@ class Category extends React.Component {
<span
role="menuitem"
tabIndex="0"
data-testclass="category-expand"
data-testid={`${metadataField}:category-expand`}
onKeyPress={(e) => {
if (e.key === "Enter") {

View File

@@ -1,6 +1,6 @@
import { glPointFlags, glPointSize } from "../../util/glHelpers";
export default function (regl) {
export default function drawPointsRegl(regl) {
return regl({
vert: `
precision mediump float;

View File

@@ -1,6 +1,6 @@
import { glPointFlags, glPointSize } from "../../util/glHelpers";
export default function (regl) {
export default function drawPointsRegl(regl) {
return regl({
vert: `
precision mediump float;

View File

@@ -1,5 +1,6 @@
import { Colors } from "@blueprintjs/core";
import { dispatchNetworkErrorMessageToUser } from "./util/actionHelpers";
import * as ENV_DEFAULT from "../../environment.default.json";
/* if a categorical metadata field has more options than this, truncate */
export const maxCategoricalOptionsToDisplay = 200;
@@ -82,21 +83,25 @@ export const maxGenes = 100;
export const tooltipHoverOpenDelay = 1000; /* ms delay before a tooltip displays */
export const tooltipHoverOpenDelayQuick = 500;
const CXG_SERVER_PORT =
process.env.CXG_SERVER_PORT || ENV_DEFAULT.CXG_SERVER_PORT;
let _API;
if (window.CELLXGENE && window.CELLXGENE.API) {
_API = window.CELLXGENE.API;
} else {
if (process.env.CXG_SERVER_PORT === undefined) {
if (CXG_SERVER_PORT === undefined) {
const errorMessage = "Please set the CXG_SERVER_PORT environment variable.";
dispatchNetworkErrorMessageToUser(errorMessage);
throw new Error(errorMessage);
}
_API = {
// prefix: "http://api.clustering.czi.technology/api/",
// prefix: "http://tabulamuris.cxg.czi.technology/api/",
// prefix: "http://api-staging.clustering.czi.technology/api/",
prefix: `http://localhost:${process.env.CXG_SERVER_PORT}/api/`,
prefix: `http://localhost:${CXG_SERVER_PORT}/api/`,
version: "v0.2/",
};
}

View File

@@ -3,11 +3,12 @@ PATH := $(PATH):$(PROJECT_ROOT)/scripts
SHELL := env PATH='$(PATH)' /bin/bash
# get_or_else_dev_env_default
# - If a variable is defined, return its value
# - Else return the default value from environment.dev
define get_or_else_dev_env_default
$(if $($(1)),$($(1)),$(shell VAR=$$(sed -n 's/$(1)=\(.*\)/\1/p' $(PROJECT_ROOT)/environment.default); eval "echo \"$$VAR\""))
ifeq ($(shell which jq),)
$(error Please install jq using "apt-get install jq" or "brew install jq")
endif
define GetValueFromJson
$(shell jq -r '.$(1)' "$(PROJECT_ROOT)/environment.default.json")
endef
# https://stackoverflow.com/a/14777895/9587410
@@ -16,11 +17,11 @@ ifeq ($(shell uname),Darwin) # is Windows_NT on XP, 2000, 7, Vista, 10...
endif
export CELLXGENE_COMMIT := $(shell git rev-parse --short HEAD)
export CXG_SERVER_PORT := $(call get_or_else_dev_env_default,CXG_SERVER_PORT)
export CXG_CLIENT_PORT := $(call get_or_else_dev_env_default,CXG_CLIENT_PORT)
export JEST_ENV := $(call get_or_else_dev_env_default,JEST_ENV)
export DATASET := $(call get_or_else_dev_env_default,DATASET)
export CXG_OPTIONS := $(call get_or_else_dev_env_default,CXG_OPTIONS)
export CXG_SERVER_PORT := $(call GetValueFromJson,CXG_SERVER_PORT)
export CXG_CLIENT_PORT := $(call GetValueFromJson,CXG_CLIENT_PORT)
export JEST_ENV := $(call GetValueFromJson,JEST_ENV)
export DATASET := $(call GetValueFromJson,DATASET)
export CXG_OPTIONS := $(call GetValueFromJson,CXG_OPTIONS)
.PHONY: start-server
start-server:

View File

@@ -1,6 +1,7 @@
# Developer guidelines
### Requirements
## Requirements
- npm
- Python 3.6+
- Chrome
@@ -11,20 +12,24 @@
### Environment
For all `make` commands, `common.mk` automatically checks whether required environment variables are set and, if they are not set, assigns them default values from `environment.default`.
For all `make` commands, `common.mk` automatically checks whether required environment variables are set and, if they are not set, assigns them default values from `environment.default.json`.
You can set these environment variables manually with the `export` shell command, as in `export JEST_ENV=debug`.
You can set these environment variables manually with the `export` shell command, as in `export JEST_ENV=debug`, or you can just pass the variables as part of the command. E.g., `HEADFUL=true make e2e` or `JEST_ENV=debug npm run e2e`
## Running test suite
Client and server tests run on Travis CI for every push, PR, and commit to master on github. End to end tests run nightly on master only.
Client and server tests run on Travis CI for every push, PR, and commit to master on github. End to end tests run nightly on master only.
### Unit tests
Steps to run the all unit tests:
1. Start in the project root directory
1. `make dev-env`
1. `make unit-test`
To run unit tests for the `client` code only:
1. Start in the project root directory
1. `cd client`
1. `make unit-test`
@@ -33,37 +38,58 @@ To run unit tests for the `client` code only:
To run E2E tests, run `cd client` and `make smoke-test`
The `JEST_ENV` environment variable enables the following E2E test options:
* `dev` - opens chromimum, runs tests with minimal slowdown, close on exit.
* `debug` - opens chromium, runs tests with 100ms slowdown, dev tools open, chrome stays open on exit.
* `prod` - run headless with no slowdown, chromium will not open.
#### Flags
Run end to end tests interactively during development
1. cellxgene should be installed as [specified in client dev](#install-1)
1. Follow [launch](#launch-1) instructions for client dev with dataset `example-dataset/pbmc3k`
1. Run `npm run e2e` or `make e2e` from the `client` directory
1. To debug a failing test `export JEST_ENV='debug'` and re-run.
1. `JEST_ENV`: This enables the following E2E test options. You can find their corresponding configs in [`jest-puppeteer.config.js`](../client/jest-puppeteer.config.js):
To run end to end tests _exactly_ as they will be run on CI use the following command:
```
- `dev` - opens window, runs tests with minimal slowdown, close on exit.
- `debug` - opens window, runs tests with 100ms slowdown, dev tools open, chrome stays open on exit.
- `prod`[default] - run headless with no slowdown, window will not open.
2. `HEADFUL`: Default is `false`. When set to `true`, it will launch the Chrome window for visual inspection. E.g., `HEADFUL=true npm run e2e`
3. `HEADLESS`: Default is `true`. When set to `false`, it will launch the Chrome window for visual inspection. E.g., `HEADLESS=false npm run e2e`
#### Run end to end tests interactively during development
1. cellxgene should be installed as [specified in client dev](#install)
1. Follow [launch](#launch) instructions for client dev with dataset `example-dataset/pbmc3k`
1. Run `npm run e2e` from the `client` directory
1. To debug a failing test, add `debugger` in any line of JS code as breakpoint, and launch the test again with [`ndb`](https://github.com/GoogleChromeLabs/ndb). E.g., `ndb make e2e` or `ndb npm run e2e`.
1. Please make sure to install `ndb` via `npm install -g ndb`
1. Check out [Debugging Tips](e2e_tests.md#debugging-tips) for more ideas!
#### To run end to end tests _exactly_ as they will be run on CI use the following command
```shell
JEST_ENV=prod make pydist install-dist dev-env smoke-test
```
## Server dev
### Install
To install from the source tree
* Build the client and put static files in place: `make build-for-server-dev`
* Install from local files: `make install-dev`
- Build the client and put static files in place: `make build-for-server-dev`
- Install from local files: `make install-dev`
To install from a candidate python distribution
* Make the distribution: `make pydist`
* Install it: `make install-dist`
- Make the distribution: `make pydist`
- Install it: `make install-dist`
### Launch
* `cellxgene launch [options] <datafile>` or `make start-server`
- `cellxgene launch [options] <datafile>` or `make start-server`
### Reloading
If you install cellxgene using `make install-dev` the server will be restarted every time you make changes on the server code. If changes affects the client, the browser must be reloaded.
### Linter
@@ -73,41 +99,55 @@ We use [`flake8`](https://github.com/PyCQA/flake8) to lint python and [`black`](
To auto-format code run `make fmt`. To run lint checks on the code run `make lint`.
### Test
If you would like to run the server tests individually, follow the steps below
1. Install development requirements `make dev-env`
1. Run `make unit-test` in the `server` directory or `make unit-test-server` in the root directory.
### Tips
* Install in a virtualenv
* May need to rebuild/reinstall when you make client changes
- Install in a virtualenv
- May need to rebuild/reinstall when you make client changes
## Client dev
### Install
1. Install prereqs for client: `make dev-env`
2. Install cellxgene server as described in the [server install](#install) instructions above.
### Launch
To launch with hot reloading, you need to launch the server and the client separately. Node's hot reloading starts the client on its own node server and auto-refreshes when changes are made to source files.
1. Launch server (the client relies on the REST API being available): `cellxgene launch --debug [other_options] <datafile>` or `make start-server`
2. Launch client: in `client/` directory run `make start-frontend`
3. Client will be served on `localhost:3000`
### Build
To build only the client: `make build-client`
### Linter
We use `eslint` to lint the code and `prettier` as our code formatter.
### Test
If you would like to run the client tests individually, follow the steps below in the `client` directory
1. For unit tests run `make unit-test`
1. For the smoke test run `make smoke-test` for the standard smoke test suite and `make smoke-test-annotations` for the annotations test suite.
If you would like to run the smoke tests against a hot-reloaded version of the client:
1. Start the hot-reloading servers as described in the [Client dev section](#client-dev). If you plan to run the standard test suite (without annotations), you'll have to start the backend server with annotations disabled (e.g. `CXG_OPTIONS='--debug --disable-annotations' make start-server`).
1. From the project root, `cd client`
1. Run either the standard E2E test suite with `CXG_CLIENT_PORT=3000 make e2e` or the annotations test suite with `CXG_CLIENT_PORT=3000 make e2e-annotations`
1. Run either the standard E2E test suite with `npm run e2e` or the annotations test suite with `npm run e2e-annotations`
### Tips
* You can also install/launch the server side code from npm scrips (requires python3.6 with virtualenv) with the `scripts/backend_dev` script.
- You can also install/launch the server side code from npm scrips (requires python3.6 with virtualenv) with the `scripts/backend_dev` script.
- Check out [e2e Tests](e2e_tests.md) for more details

View File

@@ -109,8 +109,8 @@ Methods used to test the client javascript code
`backend-dev` targets. One starts the server, the other runs the tests. If
developing a front-end feature and just checking if tests pass, this is
probabaly the one you want to run.
* `make e2e` Runs backend tests without starting the server. You will need to
* `npm run e2e` Runs backend tests without starting the server. You will need to
start the rest api separately with the pbmc3k.h5ad file. Note you can use
the `JEST_ENV` environment variable to change how JEST runs in the browser.
The test runs against `localhost:5005` by default. You can use the
The test runs against `localhost:3000` by default. You can use the
`CXG_URL_BASE` env variable to test non-localhost deployments of cellxgene.

View File

@@ -1,47 +1,88 @@
## End to end testing thesis
# End to end testing
## Thesis
End to end (e2e) tests, when done correctly, can surface errors and regressions in the application before it is released to production. The tests are run against a version of the web app that is running the full stack. By checking the fully functional app, you are able to surface issues with logic and data at every level of the stack (you still have to figure out which level that error is occurring at). You are also able to test how features that are added affect the whole application.
The major problem most developers have with e2e tests are that they the tests can be flaky and thus not trusted. Failing tests wind up being disabled instead of fixed. To guard against that we need to establish principals about what features to test and have guidelines for writing tests to ensure that they are as robust as possible. By ensuring each test has a high value and is unlikely to fail when it shouldn't it increases the motivation to fix the test as opposed to deleting it.
The major problem most developers have with e2e tests are that the tests can be flaky and thus not trusted. Failing tests wind up being disabled instead of fixed. To guard against that we need to establish principals about what features to test and have guidelines for writing tests to ensure that they are as robust as possible. By ensuring each test has a high value and is unlikely to fail when it shouldn't it increases the motivation to fix the test as opposed to deleting it.
### Choosing what to test:
Additionally, we try to combat flaky tests by re-running failed tests two more times before declaring a test is truly failing. Even though this mediation doesn't solve the root cause of the flakiness, it does help decrease the false positive cases caused by the flakiness.
* Common actions - Features and actions that are highly used will cause more impact if they break
* High value - Features that may be rarely used, but cause a major business impact if they break
* Depend on the whole stack and not just the FE - We want to test the backend and data as well as the FE. Features that only depend on the javascript code should be tested in unit tests instead.
* Feature is in a stable state - If the feature is in active development, it's likely that the test will have to be constantly updated. Wait until it is in a steady state before adding an e2e test.
* Feature is a likely target of regressions - If a feature has caused regressions in the past, it's good to add some monitoring to it.
### Choosing what to test
### Guidelines for writing tests:
- Common actions - Features and actions that are highly used will cause more impact if they break
- High value - Features that may be rarely used, but cause a major business impact if they break
- Depend on the whole stack and not just the FE - We want to test the backend and data as well as the FE. Features that only depend on the javascript code should be tested in unit tests instead.
- Feature is in a stable state - If the feature is in active development, it's likely that the test will have to be constantly updated. Wait until it is in a steady state before adding an e2e test.
- Feature is a likely target of regressions - If a feature has caused regressions in the past, it's good to add some monitoring to it.
* Use data attributes as selectors instead of positional, classname, or id selectors. Add `data-testid` or `data-testclass` attributes to components that are inputs and outputs. The advantages are 1) You can move the component around and even change it's html element type and it will not break the test and 2) It is self documenting what the attribute is used by as opposed to a class name or id that could be used by CSS or something else.
* Use the helper functions in `puppeteerUtils.js` and ` for performing common actions. Puppeteer operates at a low level and these abstract it to a level of user actions.
### Guidelines for writing tests
- Use data attributes as selectors instead of positional, classname, or id selectors. Add `data-testid` or `data-testclass` attributes to components that are inputs and outputs. The advantages are 1) You can move the component around and even change its html element type and it will not break the test and 2) It is self documenting what the attribute is used by as opposed to a class name or id that could be used by CSS or something else.
- Use the helper functions in `puppeteerUtils.js` and `cellxgeneActions.js` for performing common actions. Puppeteer operates at a low level and these abstract it to a level of user actions.
- We use [`expect-puppeteer`](https://github.com/smooth-code/jest-puppeteer/tree/master/packages/expect-puppeteer#api)'s API to avoid directly using Puppeteer's API where possible. So **please only resort to Puppeteer's API if you cannot achieve an interaction you need via `expect-puppeteer`.**
For actions that don't meet these guidelines, you can create a test in feature.test.js that does not run on PRs so if the feature breaks it doesn't block anyone.
## Where/How
Where: client/\_\_tests\_\_/e2e/e2e.test.js
How: `npm run e2e`
How: `npm run e2e` or `make e2e`
Tests are written using the Jest testing framework. Browser automation is handled by puppeteer. Puppeteer provides an API to control Chrome over the DevTools Protocol. This allows us to write code that mimics user actions and check page state.
Tests are written using the Jest testing framework. Browser automation is handled by Puppeteer. Puppeteer provides an API to control Chrome over the DevTools Protocol. This allows us to write code that mimics user actions and check page state.
Tests are running on CI with every PR, they can also be run locally (see [developer guidelines](developer_guidelines.md)).
Tests are running on CI with every PR, they can also be run locally (see [developer guidelines](developer_guidelines.md)).
The cellxgene instance is running with the pbmc3k dataset. Tests for additional datasets can be added, the data.js file contains the values for inputs and outputs expected for different datasets. Choose values that make sense for that dataset.
The cellxgene instance is running with the `pbmc3k` dataset. Tests for additional datasets can be added, the `data.js` file contains the values for inputs and outputs expected for different datasets. Choose values that make sense for that dataset.
## Setting up e2e tests locally
See [developer guidelines](developer_guidelines.md)
## How to write a new test
1. Ensure that your new test meets the guidelines above for high-value robust user actions to test.
2. Clearly define the series of user actions and outputs you want to test
3. Add testid (and/or testclass if element is not unique) data attributes to inputs and outputs you want to interact with.
4. Create a new test block possible nested in a describe block if you want to add more than one related test client/\_\_tests\_\_/e2e/e2e.test.js
5. Write the test code.
6. Run tests locally to ensure they pass.
**Debugging Tips**
* Mystery bugs are often caused by a missing `async` or `await`. Double check every function and every line (twice!).
* Switch from to debug mode by running `export JEST_ENV='debug'` on the CLI before running the tests
* Put a javascript breakpoint in by adding this line `await jestPuppeteer.debug()` to the test at the point which you want to pause execution.
* Limit the tests that are run by adding `.only` after `describe` or `test` to limit execution to just that block of tests. (Remember to take the `.only` out before you commit!)
1. Ensure that your new test meets the guidelines above for high-value robust user actions to test.
1. Clearly define the series of user actions and outputs you want to test
1. Add `testid` (and/or `testclass` if element is not unique) data attributes to inputs and outputs you want to interact with.
E.g., `<div data-testid="test-id" />` or `<div data-testclass="test-class" />`
1. Create a new test block possible nested in a describe block if you want to add more than one related test client/\_\_tests\_\_/e2e/e2e.test.js
1. Write the test code.
1. Run tests locally to ensure they pass.
## Debugging Tips
- Mystery bugs are often caused by a missing `async` or `await`. Double check every function and every line (twice!).
- Use `ndb` for debugging: Put a javascript breakpoint in by adding this line `debugger` to the test at the point which you want to pause execution. And then run `ndb npm run e2e` or `ndb make e2e`.
NOTE: You will need to global install [`ndb`](https://github.com/GoogleChromeLabs/ndb) via `npm install -g ndb`, which provides an improved debugging experience for Node.js, enabled by Chrome DevTools
- Limit the tests that are run by adding `.only` after `describe` or `test` to limit execution to just that block of tests. (Remember to take the `.only` out before you commit!)
- Avoid using `beforeEach() => {}` to set up test environment for each test, since Jest/Jasmine doesn't fail a test when a set up step fails, and we will end up getting obscure errors and don't know why a test fails. Another reason is for readability as explained [here](https://kentcdodds.com/blog/avoid-nesting-when-youre-testing/)
- Use [`takeScreenshot()`](../client/__tests__/e2e/takeScreenshot.js) to timestamp and store screenshots in the already git ignored folder `client/__tests__/screenshots/`, so you don't accidentally check in your screenshot in your commit
## Config Files
1. [jest-puppeteer.config.js](../client/jest-puppeteer.config.js) is for configuring Puppeteer's launch options as specified [here](https://github.com/puppeteer/puppeteer/blob/v3.1.0/docs/api.md#puppeteerlaunchoptions). Note: The link is to the doc for **v3.1.0**
1. [e2eJestConfig.json](../client/__tests__/e2e/e2eJestConfig.json) is for configuring `jest` when running e2e tests. For example, in `package.json` we have:
```ts
{
"e2e": "jest --config __tests__/e2e/e2eJestConfig.json e2e/e2e.test.js",
}
```
which tells `jest` to use `e2eJestConfig.json` as the config file to run e2e test file `e2e.test.js`
1. [puppeteer.setup.js](../client/__tests__/e2e/puppeteer.setup.js) is for configuring `jest`, `browser`, and `page` objects at runtime

View File

@@ -1,6 +0,0 @@
# Default environment variables for cellxgene development
CXG_SERVER_PORT=5005
CXG_CLIENT_PORT=3000
JEST_ENV=dev
DATASET="$(git rev-parse --show-toplevel)/example-dataset/pbmc3k.h5ad"
CXG_OPTIONS=--debug

10
environment.default.json Normal file
View File

@@ -0,0 +1,10 @@
{
"CXG_CLIENT_PORT": 3000,
"CXG_OPTIONS": "--debug",
"CXG_SERVER_PORT": 5005,
"DATASET": "$(git rev-parse --show-toplevel)/example-dataset/pbmc3k.h5ad",
"DEBUG": "debug",
"DEV": "dev",
"JEST_ENV": "prod",
"RETRY_ATTEMPTS": 2
}

View File

@@ -15,4 +15,3 @@ unit-test:
--start-directory test/ \
--top-level-directory ../ \
--verbose