118 Commits

Author SHA1 Message Date
Stan
a699578c34 Release v1.10.0 2026-05-01 19:34:38 +02:00
Islam Asabaev
5975a2d081 feat: add ru locale and improve translations (#320)
* Add ru locale

* fix

* fix: remove duplicate

* Add field labels and continue ru locale

* fix: Add ru gettext for queue columns, short result column, and OrmQ field labels

* fix: locale paths in translation files

* chore: format code with ruff

* chore: format code with ruff

* fix: correct time_taken usage in admin

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update django_q/locale/de/LC_MESSAGES/django.po

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update django_q/locale/tr/LC_MESSAGES/django.po

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: clear fuzzy markers

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-25 01:06:07 +02:00
Islam Asabaev
936fdd10b1 Update Python base image to 3.9-slim-bookworm (#325)
* issue: gh-324
2026-04-18 16:48:55 +02:00
Nick Yuan
fab97463c2 Fix unbounded growth of Broker.set_stat cluster master list (#322)
Prune stale entries from the master list on new cluster registration
and only write the list when membership changes (with timeout=None).

Before this fix, set_stat was append-only — dead cluster keys accumulated
forever because the only cleanup path (get_stats) runs only from the
monitor UIs. On DatabaseCache backends this rewrote the growing pickled
list to Postgres on every heartbeat, producing measurable egress.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 00:24:43 +02:00
Sophile
1abdc5b653 Convert queue size and count to string in monitor (#319) 2026-03-27 23:00:35 +01:00
Seba
01df35c923 Don't close DB connection if async_task was called with sync=True (#301) 2026-03-12 15:33:19 +01:00
Niko
573b8da528 Update Django Q2 compatibility information (#316) 2026-02-16 22:59:51 +01:00
Yibo Li
8a563d289d feat:add Simplified Chinese Translation (#314) 2026-01-14 23:58:54 +01:00
Marcin Bachry
1b0f71a39b Fix BadSignature after the default Django cache expires (#311)
* async_iter: fix BadSignature after the default Django cache expires

* lint
2026-01-08 23:27:03 +01:00
prollings
b51575a4b0 Add post_execute_in_worker signal (#309) 2025-12-10 16:54:14 +01:00
xuheng.zong
fd927087f1 fix: Fix incorrect signal import (#308)
* fix: Fix incorrect signal import

* lint

---------

Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
2025-12-08 21:10:10 +01:00
Stan
386e04171b Release v1.9.0 2025-12-04 23:04:52 +01:00
Luke Plant
0054535624 Django 6.0 support (#307)
* Remove upper bound on Django version.

This allows it to be installed alongside Django 6 and later.

See https://iscinumpy.dev/post/bound-version-constraints/

* Run tests on Django 6.0 / Python 3.12
2025-12-04 17:15:18 +01:00
Salvo 'LtWorf' Tomaselli
09e65da5b3 Fix to make tests work with redis-py > 5 (#282) 2025-07-04 02:13:46 +02:00
Stan
65d2548e98 Release v1.8.0 2025-04-25 16:30:51 +02:00
Stan Triepels
cbe51f995c Move timeout function from ORM broker into class to allow easy customization (#274)
* move timeout function to class to allow customization

* format

* fix format
2025-04-23 15:44:37 +02:00
Adam Johnson
0090a6f411 Update tested versions, add python 3.13 support and django 5.2 support. Drop python 3.8 support (#271)
* Update tested versions

* Remove f string

* Fix second `format_html`

---------

Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
2025-04-16 01:48:51 +02:00
Fernando Karchiloff
a154fd658a Avoid creating task run on Scheduler creation (#268) 2025-03-12 23:31:00 +01:00
Adam Johnson
81f2531e3e Remove disque from CI (#270) 2025-03-12 23:28:59 +01:00
Stan Triepels
c464751ae9 Delete deprecated imports and unwrapping (#261)
* delete old code

* remove imports
2025-01-12 03:40:18 +01:00
Stan
144536cbe4 Release v1.7.6 2025-01-12 02:25:29 +01:00
Stan Triepels
7e72b68c7f Make poetry version fixed in CI (#260) 2025-01-12 02:22:14 +01:00
Stan
404932117e Release v1.7.5 2025-01-12 01:57:13 +01:00
Anthony Hivert
ddc1aa2de1 perf: avoid checking success tasks when save limit is disabled (#255)
- avoid extra queries when checking the count of success in db
2025-01-04 02:31:30 +01:00
mhaehnel
351bf66d71 Fix install path for CHANGELOG.md (#258)
The file should not be installed directly in site-packages. Use the
correct include syntax to include them into the source distribution
package only.
2025-01-02 01:52:07 +01:00
Stan
ccd760a69f Release v1.7.4 2024-11-03 15:26:11 +01:00
Alberto
fb23b6b825 Fix 'receive_message_wait_time_seconds' SQS broker management (#243)
* Fix 'receive_message_wait_time_seconds' SQS broker management

* Remove socket volume from localstack container
2024-11-03 15:04:35 +01:00
P-EB
35c911080b Fix BROKER_CLASS monkeypatch in test_brokers (#239) 2024-10-21 17:51:27 +02:00
P-EB
81058fc1cf Decrease the MAX_RSS set in test_cluster::test_max_rss (#240) 2024-10-21 17:50:30 +02:00
Stan
505eb8d537 Release v1.7.3 2024-10-15 04:17:28 +02:00
blockisec
dcfe5b6650 Fix missing ack_id when finishing task (#224)
* Update monitor.py

* format

---------

Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
2024-10-11 15:03:18 +02:00
Paulo Coutinho
c3b49cca25 Only trigger prometheus if configured (#231)
* fix prometheus integration

* fix prometheus integration

* Add also lowercase option

As they seem to support that too: d7c9cd88c7/prometheus_client/multiprocess.py (L166-L167)

* format

---------

Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
2024-10-11 04:09:17 +02:00
Knuth
c1e00b09cb Fix type check for args in scheduler.py E721 (#233) 2024-10-10 16:10:43 +02:00
Knuth
8e403d0136 Refactor timeout handling to handle AttributeError and ValueError for Windows users (#234) 2024-10-10 16:09:57 +02:00
Stan Triepels
337a5782d9 Catch missing SEGALRM with AttributeError instead of ValueError (#223) 2024-09-22 04:12:38 +02:00
Stan
2c456cd0aa Release v1.7.2 2024-09-10 00:16:18 +02:00
Stan Triepels
8f54e5d5ae Add twine check and fix readme (#216)
* Add twine check

* add poetry to install

* move up remove black/flake formatting

* fix path

* fix?

* fix rst?

* add rstcheck

* test all files?

* all

* remove complaining image

* only check readme

* remove not needed dependencies
2024-09-10 00:10:14 +02:00
Stan
689c881178 Release v1.7.1 2024-09-08 04:20:13 +02:00
Stan
923c5daa73 Release v1.7.0 2024-09-08 03:52:37 +02:00
Stan Triepels
69e794f1af Add index on succeeded tasks (#164)
* Add index on task table

* also add models file

* add make commands and limit index

* format
2024-09-08 03:37:28 +02:00
Stan Triepels
2191897825 Add example project (#215)
* add example project

* format

* fix format
2024-09-08 03:13:49 +02:00
Andrii Yurchuk
af505e8d70 Call mark_process_dead on worker pid if prometheus_client is installed (#212) 2024-09-03 22:29:00 +02:00
kimihito
48123af09d chore: Add Support Django 5.1 (#207) 2024-08-13 14:14:40 +02:00
kimihito
faaa57be93 chore: Update CI for Django 4.2 Python 3.12 support (#208) 2024-08-13 14:14:11 +02:00
jv3ga
5015d8ee4d fix: Oracle ORM backend compatibility #180 (#186) 2024-08-13 04:40:54 +02:00
Stan Triepels
cb253357fb Fix repeating task after timeout (#184) 2024-06-23 22:36:29 +02:00
GDay
efa0d6d9e4 ignore fuff formating 2024-05-11 03:06:58 +02:00
Stan Triepels
ad4d24e17c Replace black/isort with ruff (#188)
* Replace black/isort with ruff

* fix typo
2024-05-11 03:04:32 +02:00
Stan Triepels
1d1d0a9aa5 Fix max attempts for value 1 (#185) 2024-05-06 17:28:27 +02:00
Stan Triepels
5e4aa65f62 Remove support for Django 3.2 and 4.1 (#183) 2024-05-05 00:42:09 +02:00
GDay
7add68f770 Release v1.6.2 2024-03-05 16:46:08 +01:00
Stan Triepels
782b1de60d Allow different broker on chain (#156) 2024-03-02 22:45:40 +01:00
Josh Thomas
88430c9014 Fix formatting issues in README.rst (#159) 2024-02-20 18:04:46 +01:00
Stan Triepels
6265a54008 Update docs to add cluster option to the async_task (#157) 2024-02-18 04:00:07 +01:00
Stan Triepels
1b638e5802 Update release/test dependencies (#147)
* Update release dependencies

* Update test.yml
2023-12-14 16:06:34 +01:00
Tobias Menzel
13a382cca2 Fix for Negative Repeat Count in Scheduler (#101) (#146)
* Comment out decrement of s.repeats in scheduler to fix negative repeat count issue

* Fix repeat count handling in scheduler

- Reset `repeats` to -1 for values less than -1
- Decrement `repeats` by 1 if it's greater than 0
2023-12-14 14:44:07 +01:00
Stan Triepels
098aa66a2d Update django.po (#138) 2023-11-04 19:52:43 +01:00
Stan Triepels
b3f494571f Use importerror for b62_decode and avoid deprecation notification (#134) 2023-10-26 14:29:13 +02:00
Patrycja
503127233f Specify build system in pyproject.toml (#131)
Otherwise, trying to build the project with a tool like gpep517 or build
results in using the setuptools legacy backend instead
2023-10-19 22:05:16 +02:00
GDay
e3be22ac63 Release v1.6.1 2023-10-13 14:15:58 +02:00
Stan Triepels
0aaeb8d35d Fix strict versions for python/django - poetry install error (#130) 2023-10-13 14:13:22 +02:00
GDay
750c654940 Release v1.6.0 2023-10-13 00:59:42 +02:00
Stan Triepels
1d2a6059db Add support for Django 5 (#120) 2023-10-12 22:44:19 +02:00
Stan Triepels
54ab399758 Fix for "apps not ready" in Windows and Mac (#116) 2023-10-09 22:04:32 +02:00
Grayknife
bc50219141 Update broken MongoClient link in Docs (#127) 2023-10-07 22:15:49 +02:00
Grayknife
939d0ebef1 Fix German Translation Typo (#124) 2023-09-30 13:28:42 +02:00
Jrog
68af2ae13e Update Add-ons install command in install.rst (#115) 2023-09-14 14:37:12 +02:00
Dustin Blanchard
960f72a1d0 DOCS: Correct health check import in examples.rst (#110)
* Correct health check import in examples.rst

* Update monitor.rst

---------

Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
2023-09-05 17:05:47 +02:00
GDay
aa79a6e6b3 Release v1.5.5 2023-09-01 16:36:00 +02:00
Stan Triepels
59d761b574 Add documentation to migrate from django-q to django-q2 (#108) 2023-09-01 14:33:26 +02:00
Stan Triepels
d8cc0cc4f4 Fix not picking up result from falsy result (#107)
* Fix not picking up result from falsy result

* add test and docs

* typo
2023-09-01 02:36:15 +02:00
Rémy Pecqueur
a08ef61ba5 Remove deprecated usage pkg_resources (#103)
List entry points using importlib.metadata (Python >=3.10) or
importlib-metadata>=3.6 (Python < 3.10)
2023-08-01 02:33:31 +02:00
Stan Triepels
7fb7fca176 Move worker, scheduler, pusher and monitor to separate files (#100) 2023-07-02 02:47:05 +02:00
GDay
f8501bfb4c Release v1.5.4 2023-06-29 01:03:57 +02:00
Scott Pashley
23c06a38eb Rerun successful tasks (#99)
* Move logic that resubmits a task and then deletes the original into an action function that can be used for both failure and success resubmission.

* Use the resubmit_task action in the FailAdmin.

* Add the resubmit_task action to the TaskAdmin class.

* Update resubmit failure test to reflect using the new resubmit action.

* Add a test for resubmitting successful tasks.

---------

Co-authored-by: Scott Pashley
2023-06-23 23:05:04 +02:00
GDay
f8321e7f54 Release v1.5.3 2023-05-14 03:22:34 +02:00
Stan Triepels
630b3c62b3 Make processes identifiable with uuid4 (#91)
* make processes identifiable

* update docs
2023-05-14 03:07:38 +02:00
Stan Triepels
227caa7446 Post spawn docs (#95) 2023-05-14 03:04:07 +02:00
David Hoover
4ba9e0338a Add post_spawn signal. (#93)
Co-authored-by: David Hoover <hooverd@arizona.edu>
2023-05-14 02:51:48 +02:00
GDay
fb960104c3 Release v1.5.2 2023-04-13 01:31:59 +02:00
GDay
88bbfd8967 Mention support for 4.2.x in docs 2023-04-13 01:21:46 +02:00
Martijn Jacobs
1f31725f43 Added Django 4.2 to the test matrix, fixed deprecation warning (#89)
* Added Django 4.2 to the CI test matrix

* Positional arguments with TimestampSigner are deprecated
2023-04-12 18:18:03 +02:00
GDay
322a53f06d Release v1.5.1 2023-04-02 17:53:23 +02:00
GDay
77821b1de7 Fix release workflow after name change 2023-04-02 17:51:17 +02:00
GDay
38d48231d6 Release v1.5.0 2023-04-02 17:43:17 +02:00
Stan Triepels
d13e9d3d64 Update dependencies (2023-04-02) (#88) 2023-04-02 17:15:39 +02:00
sinowood
52c04217e9 Multiple queue, multiple cluster in one site (#71)
* Support for multi-queue, multi-cluster configuration. API changes include:
  * Adding `cluster` to async_task() parameters and Task model
  * Adding argument --name to qcluster command
  * Necessary adjustments to Conf and Broker classes
  * Some admin improvements

* Add settings.Q_CLUSTER['ALT_CLUSTERS']: q_cluster config overrides for alternative clusters;
Add Conf.CLUSTER_NAME: separate usage from Conf.PREFIX;
QueueAdmin/OrmQ detail page enhanced: now displaying args/kwargs/q_options instead of encrypted payload.

* if `cluster` argument is not set (the default), async_task() and schedule() will be handled by the default cluster; Documentation update.

* Text cleanup

* Documentation update.

* Fix TIMEOUT setting in Windows for non-default cluster

---------

Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
2023-04-02 16:36:17 +02:00
Stan Triepels
f8520c9cda Remove version locking on poetry_core to fix regex error (#87) 2023-03-31 02:43:48 +02:00
GDay
86467dcf03 Revert "Merge pull request #15 from django-q2/error-on-timeout"
This reverts commit d97dca5bcb, reversing
changes made to 01cb652071.
2023-03-31 02:22:38 +02:00
Stan Triepels
69a40344f0 Allow building docs all formats (#77) 2023-02-23 01:20:19 +01:00
Jim Jagielski
d97dca5bcb Merge pull request #15 from django-q2/error-on-timeout
Error on timeout
2023-02-17 12:41:27 -05:00
Jim Jagielski
f5ecd337ad Merge branch 'master' into error-on-timeout 2023-02-17 12:33:47 -05:00
GDay
01cb652071 Release v1.4.11 2023-01-31 00:24:44 +01:00
Stan Triepels
b9bdd64568 Remove custom build (revert to auto create setup file) (#70) 2023-01-31 00:21:37 +01:00
Stan Triepels
b2bcffbcab Fix missing setup file for "No matching distribution" error (#69) 2023-01-31 00:09:33 +01:00
GDay
25cd180410 Release v1.4.10 2023-01-27 02:10:20 +01:00
Stan Triepels
f9e01699c5 Adding translation mo files automatically on build (#65) 2023-01-26 16:45:44 +01:00
Stan Triepels
2f3b0d0a31 Update all dependencies (#64) 2023-01-26 02:56:20 +01:00
Stan Triepels
264607f59c Bump translations to latest changes (#63) 2023-01-26 02:49:46 +01:00
msabatier
17c1609f10 Add meaningfull process titles with currently running task name (#57)
* Customize process names (in ps/top) and add task name in logs

* Increase severity of reincarnate log messages and add task name

* Document setproctitle optional dependency

Co-authored-by: Marc Sabatier <37879561+msabatier@users.noreply.github.com>
Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
2023-01-26 02:21:56 +01:00
msabatier
650c3b1524 Fix use of database router for write queries and remove Conf.HAS_REPLICA (#61)
Co-authored-by: Marc Sabatier
2023-01-18 02:12:13 +01:00
msabatier
a013591de5 Add intended_date_kwarg field to Schedule (#62)
Co-authored-by: Marc Sabatier <marc.sabatier@eco-adapt.com>
2023-01-17 22:52:11 +01:00
msabatier
865b1a5ba8 Change task timeout logic to have now() as execution time (#58) 2023-01-11 01:48:57 +01:00
msabatier
8cd1028391 More explicit log messages in exception handling (#59) 2023-01-11 00:36:35 +01:00
GDay
31e82ad028 Release v1.4.9 2022-12-22 03:06:01 +01:00
Stan Triepels
b66dfbd3af Fix DST timezone change (move from DST to normal jump) (#56) 2022-12-22 03:02:53 +01:00
GDay
08baaab125 Release v1.4.8 2022-12-21 03:46:01 +01:00
Stan Triepels
ce81059da8 Fix: allow both ZoneInfo and Pytz depending on django version (#55) 2022-12-21 03:42:38 +01:00
Stan Triepels
cf33275d92 Release v1.4.7 (#54) 2022-12-21 01:53:10 +01:00
Stan Triepels
e78e473be3 Fix: handling exceptions inside job function (#51) 2022-12-21 01:44:59 +01:00
Stan Triepels
4d454e4001 Fix: Daylight saving time issue with scheduler (#47) 2022-12-21 01:44:34 +01:00
Stan Triepels
cf33891291 Chore: Remove release drafter (#53) 2022-12-21 01:44:10 +01:00
Stan Triepels
134a54dbeb Chore: Fix badge and add download badge (#52) 2022-12-21 01:43:51 +01:00
Stan Triepels
c1d4858f62 Release 1.4.6 (#43) 2022-11-30 15:09:49 +01:00
Andreas Andersen
2566955782 Log exceptions with logger.exception (#42)
Co-authored-by: Andreas Bok Andersen <andreas.bokandersen@glooko.com>
2022-11-30 15:03:20 +01:00
Stan Triepels
3c0c32758d Chore: Flake8, isort and Black (#40) 2022-11-16 01:45:40 +01:00
Noortheen Raja
64d73a77b7 feat: option to fail-on-timeout
fixes #251
2022-10-19 02:27:12 +02:00
Noortheen Raja
596842e692 chore: pip install support
having build-system config helps installing  this without setup.py built
2022-10-19 02:27:10 +02:00
101 changed files with 7288 additions and 3181 deletions

4
.git-blame-ignore-revs Normal file
View File

@@ -0,0 +1,4 @@
# flake8, black, isort
b1d000d007f3f77069719523268a0c6256dc0860
# move to ruff formatting/linting
ad4d24e17c9424b17cd8ae65c2def7ecc74e63c1

View File

@@ -1,37 +0,0 @@
categories:
-
label: breaking
title: Breaking
-
label: feature
title: New
-
label: bug
title: "Bug Fixes"
-
label: dependencies
title: "Dependency Updates"
-
label: security
title: Security
name-template: v$NEXT_PATCH_VERSION
tag-template: v$NEXT_PATCH_VERSION
template: |
# Changes
$CHANGES
version-resolver:
major:
labels:
- breaking
- major
minor:
labels:
- feature
- minor
patch:
labels:
- bug
- dependencies
- security
- patch
default: patch

View File

@@ -7,19 +7,18 @@ on:
jobs:
build:
if: github.repository == 'GDay/django-q2'
if: github.repository == 'django-q2/django-q2'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: 3.11
- name: Install dependencies
run: |
sudo apt-get update

View File

@@ -1,14 +0,0 @@
name: Update release draft
on:
push:
branches:
- master
jobs:
update_release_draft:
runs-on: ubuntu-latest
steps:
- uses: release-drafter/release-drafter@v5
with:
config-name: release-drafter.yml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -7,18 +7,58 @@ on:
branches:
- master
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Lint with ruff
run: |
pipx install ruff==0.4.10
ruff format . --check && ruff check .
- name: Check twine
run: |
python -m pip install twine poetry rstcheck
poetry build
rstcheck README.rst
twine check dist/*
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [ "3.8", "3.9", "3.10", "3.11" ]
django: [ "3.2", "4.1" ]
python-version:
- "3.9"
- "3.10"
- "3.11"
- "3.12"
django:
- "4.2"
- "5.0"
- "5.1"
- "5.2"
- "6.0"
exclude:
# django 5.2 does not support 3.9
- python-version: "3.9"
django: "5.2"
# django 5.1 does not support 3.9
- python-version: "3.9"
django: "5.1"
# django 5.0 does not support 3.9
- python-version: "3.9"
django: "5.0"
# django 4.2 does not support 3.13
- python-version: "3.13"
django: "4.2"
# django 6.0 does not support earlier than 3.12
- python-version: "3.9"
django: "6.0"
- python-version: "3.10"
django: "6.0"
- python-version: "3.11"
django: "6.0"
services:
disque:
image: efrecon/disque:1.0-rc1
ports:
- '7711:7711/tcp'
mongodb:
image: mongo
ports:
@@ -39,16 +79,16 @@ jobs:
- 6379:6379
options: --entrypoint redis-server
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies with Django ${{ matrix.django }}
run: |
python -m pip install --upgrade pip
pip install poetry
poetry add "django==${{ matrix.django }}"
pip install poetry==1.8.5
poetry add "django~=${{ matrix.django }}a1" --python=${{ matrix.python-version }}
poetry install -E testing
- name: Run Tests
run: |
@@ -68,7 +108,7 @@ jobs:
finish:
needs: test
runs-on: ubuntu-latest
container: python:3-slim
container: python:3.11-bookworm
steps:
- name: Upload to coveralls
run: |

View File

@@ -15,6 +15,8 @@ build:
sphinx:
configuration: docs/conf.py
formats: all
python:
install:
- requirements: docs/requirements.txt

View File

@@ -1,6 +1,200 @@
# Changelog
## [Unreleased](https://github.com/GDay/django-q2/tree/HEAD)
## [v1.10.0](https://github.com/django-q2/django-q2/tree/v1.10.0) (2026-05-01)
- fix: Fix incorrect signal import (#308) https://github.com/django-q2/django-q2/pull/308
- Add post_execute_in_worker signal (#309) https://github.com/django-q2/django-q2/pull/309
- Fix BadSignature after the default Django cache expires (#311) https://github.com/django-q2/django-q2/pull/311
- feat:add Simplified Chinese Translation (#314) https://github.com/django-q2/django-q2/pull/314
- Update Django Q2 compatibility information (#316) https://github.com/django-q2/django-q2/pull/316
- Don't close DB connection if async_task was called with sync=True (#301) https://github.com/django-q2/django-q2/pull/301
- Convert queue size and count to string in monitor (#319) https://github.com/django-q2/django-q2/pull/319
- Fix unbounded growth of Broker.set_stat cluster master list (#322) https://github.com/django-q2/django-q2/pull/322
- Update Python base image to 3.9-slim-bookworm (#325) https://github.com/django-q2/django-q2/pull/325
- feat: add ru locale and improve translations (#320) https://github.com/django-q2/django-q2/pull/320
## [v1.9.0](https://github.com/django-q2/django-q2/tree/v1.9.0) (2025-12-04)
- Django 6.0 support https://github.com/django-q2/django-q2/pull/307
- Fix to make tests work with redis-py > 5 https://github.com/django-q2/django-q2/pull/282
## [v1.8.0](https://github.com/django-q2/django-q2/tree/v1.8.0) (2025-04-25)
- Delete deprecated imports and unwrapping https://github.com/django-q2/django-q2/pull/261
- Remove disque from CI https://github.com/django-q2/django-q2/pull/270
- Avoid creating task run on Scheduler creation https://github.com/django-q2/django-q2/pull/268
- Update tested versions, add python 3.13 support and django 5.2 support. Drop python 3.8 support https://github.com/django-q2/django-q2/pull/271
- Move timeout function from ORM broker into class to allow easy customization https://github.com/django-q2/django-q2/pull/274
## [v1.7.6](https://github.com/django-q2/django-q2/tree/v1.7.6) (2025-01-12)
- Make poetry version fixed in CI https://github.com/django-q2/django-q2/pull/260
## [v1.7.5](https://github.com/django-q2/django-q2/tree/v1.7.5) (2025-01-12)
- perf: avoid checking success tasks when save limit is disabled https://github.com/django-q2/django-q2/pull/255
- Fix install path for CHANGELOG.md https://github.com/django-q2/django-q2/pull/258
## [v1.7.4](https://github.com/django-q2/django-q2/tree/v1.7.4) (2024-11-03)
- Decrease the MAX_RSS set in test_cluster::test_max_rss https://github.com/django-q2/django-q2/pull/240
- Fix BROKER_CLASS monkeypatch in test_brokers https://github.com/django-q2/django-q2/pull/239
- Fix 'receive_message_wait_time_seconds' SQS broker management https://github.com/django-q2/django-q2/pull/243
## [v1.7.3](https://github.com/django-q2/django-q2/tree/v1.7.3) (2024-10-15)
- Catch missing SEGALRM with AttributeError instead of ValueError https://github.com/django-q2/django-q2/pull/223
- Refactor timeout handling to handle AttributeError and ValueError for Windows users https://github.com/django-q2/django-q2/pull/234
- Fix type check for args in scheduler.py E721 https://github.com/django-q2/django-q2/pull/233
- Only trigger prometheus if configured https://github.com/django-q2/django-q2/pull/231
- Fix missing ack_id when finishing task https://github.com/django-q2/django-q2/pull/224
## [v1.7.2](https://github.com/django-q2/django-q2/tree/v1.7.2) (2024-09-09)
- Fix twine check
## [v1.7.1](https://github.com/django-q2/django-q2/tree/v1.7.1) (2024-09-08)
- Fixed date of v1.7.0
- Fixed README.rst formatting which is blocking release of latest version
## [v1.7.0](https://github.com/django-q2/django-q2/tree/v1.7.0) (2024-09-08)
**Merged pull requests:**
- Remove support for Django 3.2 and 4.1 https://github.com/django-q2/django-q2/pull/183
- Fix max attempts for value 1 https://github.com/django-q2/django-q2/pull/185
- Replace black/isort with ruff https://github.com/django-q2/django-q2/pull/188
- Fix repeating task after timeout https://github.com/django-q2/django-q2/pull/184
- fix: Oracle ORM backend compatibility #180 https://github.com/django-q2/django-q2/pull/186
- chore: Update CI for Django 4.2 Python 3.12 support https://github.com/django-q2/django-q2/pull/208
- chore: Add Support Django 5.1 https://github.com/django-q2/django-q2/pull/207
- Call mark_process_dead on worker pid if prometheus_client is installed https://github.com/django-q2/django-q2/pull/212
- Add example project https://github.com/django-q2/django-q2/pull/215
- Add index on succeeded tasks https://github.com/django-q2/django-q2/pull/164
## [v1.6.2](https://github.com/django-q2/django-q2/tree/v1.6.2) (2024-03-05)
**Merged pull requests:**
- Allow different broker on chain https://github.com/django-q2/django-q2/pull/156
- Fix formatting issues in README.rst https://github.com/django-q2/django-q2/pull/159
- Update docs to add cluster option to the async_task https://github.com/django-q2/django-q2/pull/157
- Update release/test dependencies https://github.com/django-q2/django-q2/pull/147
- Fix for Negative Repeat Count in Scheduler https://github.com/django-q2/django-q2/pull/146
- Update django.po https://github.com/django-q2/django-q2/pull/138
- Use importerror for b62_decode and avoid deprecation notification https://github.com/django-q2/django-q2/pull/134
- Specify build system in pyproject.toml https://github.com/django-q2/django-q2/pull/131
## [v1.6.1](https://github.com/django-q2/django-q2/tree/v1.6.1) (2023-10-13)
**Merged pull requests:**
- Fix strict versions for python/django https://github.com/django-q2/django-q2/pull/130
## [v1.6.0](https://github.com/django-q2/django-q2/tree/v1.6.0) (2023-10-12)
**Merged pull requests:**
- Add support for Django 5 and python 12 https://github.com/django-q2/django-q2/pull/120
- Fix for "apps not ready" in Windows and Mac https://github.com/django-q2/django-q2/pull/116
- Update broken MongoClient link in Docs https://github.com/django-q2/django-q2/pull/127
- Fix German Translation Typo https://github.com/django-q2/django-q2/pull/124
- Update Add-ons install command in install.rst https://github.com/django-q2/django-q2/pull/115
- DOCS: Correct health check import in examples.rst https://github.com/django-q2/django-q2/pull/110
## [v1.5.5](https://github.com/django-q2/django-q2/tree/v1.5.5) (2023-09-01)
**Merged pull requests:**
- Add documentation to migrate from django-q to django-q2 https://github.com/django-q2/django-q2/pull/108
- Fix not picking up result from falsy result https://github.com/django-q2/django-q2/pull/107
- Remove deprecated usage pkg_resources https://github.com/django-q2/django-q2/pull/103
- Move worker, scheduler, pusher and monitor to separate files https://github.com/django-q2/django-q2/pull/100
## [v1.5.4](https://github.com/GDay/django-q2/tree/v1.5.4) (2023-06-29)
**Merged pull requests:**
- Rerun successful tasks https://github.com/django-q2/django-q2/pull/99
## [v1.5.3](https://github.com/GDay/django-q2/tree/v1.5.3) (2023-05-14)
**Merged pull requests:**
- Add post_spawn signal. https://github.com/django-q2/django-q2/pull/93
- Post spawn docs https://github.com/django-q2/django-q2/pull/95
- Make processes identifiable with uuid4 https://github.com/django-q2/django-q2/pull/91
## [v1.5.2](https://github.com/GDay/django-q2/tree/v1.5.2) (2023-04-13)
**Merged pull requests:**
- Added Django 4.2 to the test matrix, fixed deprecation warning https://github.com/GDay/django-q2/pull/89
- Updated docs to show support for 4.2
## [v1.5.1](https://github.com/GDay/django-q2/tree/v1.5.1) (2023-04-02)
- Fix release to pipy due to changed org name
## [v1.5.0](https://github.com/GDay/django-q2/tree/v1.5.0) (2023-04-02)
**Merged pull requests:**
- Multiple queue, multiple cluster in one site https://github.com/GDay/django-q2/pull/71
- Allow building docs all formats https://github.com/GDay/django-q2/pull/77
- Remove version locking on `poetry_core` to fix regex error https://github.com/GDay/django-q2/pull/87
- Update dependencies (2023-04-02) https://github.com/GDay/django-q2/pull/88
## [v1.4.11](https://github.com/GDay/django-q2/tree/v1.4.11) (2023-01-30)
**Merged pull requests:**
- Fix missing setup file for "No matching distribution" error https://github.com/GDay/django-q2/pull/69
- Remove custom build (revert to auto create setup file) https://github.com/GDay/django-q2/pull/70
## [v1.4.10](https://github.com/GDay/django-q2/tree/v1.4.10) (2023-01-26)
**Merged pull requests:**
- Adding translation mo files automatically on build https://github.com/GDay/django-q2/pull/65
- Update all dependencies https://github.com/GDay/django-q2/pull/64
- Bump translations to latest changes https://github.com/GDay/django-q2/pull/63
- Add meaningfull process titles with currently running task name https://github.com/GDay/django-q2/pull/57
- Fix use of database router for write queries and remove Conf.HAS_REPLICA https://github.com/GDay/django-q2/pull/61
- Add intended_date_kwarg field to Schedule https://github.com/GDay/django-q2/pull/62
- Change task timeout logic to have now() as execution time https://github.com/GDay/django-q2/pull/58
- More explicit log messages in exception handling https://github.com/GDay/django-q2/pull/59
## [v1.4.9](https://github.com/GDay/django-q2/tree/v1.4.9) (2022-12-22)
**Merged pull requests:**
- Fix DST timezone change (move from DST to normal jump) https://github.com/GDay/django-q2/pull/56
## [v1.4.8](https://github.com/GDay/django-q2/tree/v1.4.8) (2022-12-21)
**Merged pull requests:**
- Fix: allow both ZoneInfo and Pytz depending on django version https://github.com/GDay/django-q2/pull/55
## [v1.4.7](https://github.com/GDay/django-q2/tree/v1.4.7) (2022-12-21)
**Merged pull requests:**
- Fix: handling exceptions inside job function https://github.com/GDay/django-q2/pull/51
- Fix: Daylight saving time issue with scheduler https://github.com/GDay/django-q2/pull/47
- Chore: Fix badge and add download badge https://github.com/GDay/django-q2/pull/52
- Chore: Remove release drafter https://github.com/GDay/django-q2/pull/53
## [v1.4.6](https://github.com/GDay/django-q2/tree/v1.4.6) (2022-11-30)
**Merged pull requests:**
- Fix: Log exceptions with logger.exception https://github.com/GDay/django-q2/pull/42
- Chore: flake8, isort, black https://github.com/GDay/django-q2/pull/40
## [v1.4.5](https://github.com/GDay/django-q2/tree/v1.4.5) (2022-11-13)

View File

@@ -1,4 +1,4 @@
FROM python:3.9
FROM python:3.12
ENV PYTHONUNBUFFERED 1
RUN mkdir -p /app

View File

@@ -1,5 +1,5 @@
# Sets the python version
FROM python:3.9.5-slim
FROM python:3.9-slim-bookworm
# Allows the logs generated by python apps to be rendered in the terminal
ENV PYTHONUNBUFFERED 1
@@ -7,11 +7,15 @@ ENV PYTHONUNBUFFERED 1
# Sets the default shell to bash
ENV SHELL /bin/bash
RUN set -ex \
&& apt update \
&& apt-get install gcc python3-dev --yes
# Upgrades pip
RUN pip install --upgrade pip
RUN pip install -U pip setuptools
# Install poetry
RUN pip install poetry
RUN pip install poetry==1.8.2
WORKDIR /app

21
Makefile Normal file
View File

@@ -0,0 +1,21 @@
dev:
docker compose -f web-docker-compose.yaml up
test:
docker compose -f test-services-docker-compose.yaml run --rm django-q2 poetry run pytest
shell:
docker compose -f test-services-docker-compose.yaml run --rm django-q2 poetry run python manage.py shell
makemigrations:
docker compose -f test-services-docker-compose.yaml run --rm django-q2 poetry run python manage.py makemigrations
migrate:
docker compose -f test-services-docker-compose.yaml run --rm django-q2 poetry run python manage.py migrate
createsuperuser:
docker compose -f web-docker-compose.yaml run --rm web python manage.py createsuperuser
format:
docker compose -f test-services-docker-compose.yaml run --rm django-q2 poetry run ruff format .
docker compose -f test-services-docker-compose.yaml run --rm django-q2 poetry run ruff check . --fix

View File

@@ -1,11 +1,9 @@
A multiprocessing distributed task queue for Django
---------------------------------------------------
|image0| |image1| |docs|
|image0| |image1| |downloads|
::
Django Q2 is a fork of Django Q. Big thanks to Ilan Steemers for starting this project. Unfortunately, development has stalled since June 2021. Django Q2 is the new updated version of Django Q, with dependencies updates, docs updates and several bug fixes. Original repository: https://github.com/Koed00/django-q
Django Q2 is a fork of Django Q. Big thanks to Ilan Steemers for starting this project. Unfortunately, development has stalled since June 2021. Django Q2 is the new updated version of Django Q, with dependencies updates, docs updates and several bug fixes. Original repository: https://github.com/Koed00/django-q
Features
~~~~~~~~
@@ -27,7 +25,7 @@ Changes compared to the original Django-Q:
- Dropped support for Disque (hasn't been updated in a long time)
- Dropped Redis, Arrow and Blessed dependencies
- Updated all current dependencies
- Added tests for Django 4.x
- Added tests for Django 4.x and 5.x
- Added Turkish language
- Improved admin area
- Fixed a lot of issues
@@ -37,10 +35,13 @@ See the `changelog <https://github.com/GDay/django-q2/blob/master/CHANGELOG.md>`
Requirements
~~~~~~~~~~~~
- `Django <https://www.djangoproject.com>`__ > = 3.2
- `Django <https://www.djangoproject.com>`__ > = 4.2
- `Django-picklefield <https://github.com/gintas/django-picklefield>`__
Tested with: Python 3.8, 3.9, 3.10, 3.11 Django 3.2.X and 4.1.X
Tested with:
* Python 3.9 to 3.13.
* Django 4.2 to 6.0.
Brokers
~~~~~~~
@@ -73,7 +74,6 @@ Installation
Read the full documentation at `https://django-q2.readthedocs.org <https://django-q2.readthedocs.org>`__
Configuration
~~~~~~~~~~~~~
@@ -104,10 +104,7 @@ For full configuration options, see the `configuration documentation <https://dj
Management Commands
~~~~~~~~~~~~~~~~~~~
::
For the management commands to work, you will need to install Blessed: <https://github.com/jquast/blessed>
For the management commands to work, you will need to install Blessed: <https://github.com/jquast/blessed>
Start a cluster with::
@@ -159,7 +156,6 @@ Use `async_task` from your code to quickly offload tasks:
For more info see `Tasks <https://django-q2.readthedocs.org/en/latest/tasks.html>`__
Schedule
~~~~~~~~
@@ -204,6 +200,22 @@ Admin page or directly from your code:
For more info check the `Schedules <https://django-q2.readthedocs.org/en/latest/schedules.html>`__ documentation.
Development
~~~~~~~~~~~
There is an example project that you can use to develop with. Docker (compose) is being used to set everything up.
Please note that you will have to restart the django-q container when changes have been made to tasks or django-q.
You can start the example project with:
.. code:: bash
make dev
Create a superuser with:
.. code:: bash
make createsuperuser
Testing
~~~~~~~
@@ -212,7 +224,7 @@ Running tests is easy with docker compose, it will also start the necessary data
.. code:: bash
docker-compose -f test-services-docker-compose.yaml run --rm django-q2 poetry run pytest
make test
Locale
~~~~~~
@@ -220,12 +232,6 @@ Locale
Currently available in English, German, Turkish, and French.
Translation pull requests are always welcome.
Todo
~~~~
- Better tests and coverage
- Less dependencies?
Acknowledgements
~~~~~~~~~~~~~~~~
@@ -242,7 +248,5 @@ Acknowledgements
:target: https://github.com/GDay/django-q2/actions?query=workflow%3Atests
.. |image1| image:: https://coveralls.io/repos/github/GDay/django-q2/badge.svg?branch=master
:target: https://coveralls.io/github/GDay/django-q2?branch=master
.. |docs| image:: https://readthedocs.org/projects/docs/badge/?version=latest
:alt: Documentation Status
:scale: 100
:target: https://django-q.readthedocs.org/
.. |downloads| image:: https://img.shields.io/pypi/dm/django-q2
:target: https://img.shields.io/pypi/dm/django-q2

19
containers/localstack/init.sh Executable file
View File

@@ -0,0 +1,19 @@
#!/bin/bash
# Note that this file needs to have the executable bit set for it to work with later localstack implementations.
export DEFAULT_REGION=us-west-2
create_sqs() {
QUEUE_NAME="$1"
TIMEOUT=${2:-60}
DL_QUEUE_URL=$(awslocal sqs create-queue --queue-name "dl-$QUEUE_NAME" --query QueueUrl --output text)
echo ">>> Created $DL_QUEUE_URL queue!"
DL_QUEUE_ARN=$(awslocal sqs get-queue-attributes --queue-url "$DL_QUEUE_URL" --attribute-names QueueArn --query Attributes.QueueArn --output text)
awslocal sqs create-queue --queue-name "$QUEUE_NAME" --attributes '{
"RedrivePolicy": "{\"deadLetterTargetArn\": \"'"$DL_QUEUE_ARN"'\",\"maxReceiveCount\":\"3\"}",
"VisibilityTimeout": "'"$TIMEOUT"'"
}'
}
# Create SQS queues
create_sqs testing

View File

@@ -0,0 +1,9 @@
import subprocess
def generate_mo_files():
subprocess.run(["django-admin", "compilemessages"])
if __name__ == "__main__":
generate_mo_files()

View File

@@ -1,8 +1,3 @@
VERSION = (1, 3, 9)
import django
if django.VERSION < (3, 2):
default_app_config = "django_q.apps.DjangoQConfig"
VERSION = (1, 10, 0)
__all__ = ["conf", "cluster", "models", "tasks"]

View File

@@ -1,19 +1,48 @@
"""Admin module for Django."""
from django.contrib import admin
from django.db.models.expressions import OuterRef, Subquery
from django.template.defaultfilters import truncatechars
from django.urls import reverse
from django.utils.html import format_html
from django.contrib import admin
from django.utils.translation import gettext_lazy as _
from django.db.models.expressions import OuterRef, Subquery
from django_q.conf import Conf, croniter
from django_q.models import Failure, OrmQ, Schedule, Success, Task
from django_q.tasks import async_task
def resubmit_task(model_admin, request, queryset):
"""Submit selected tasks back to the queue."""
for task in queryset:
async_task(
task.func,
*task.args or (),
hook=task.hook,
group=task.group,
cluster=task.cluster,
**task.kwargs or {},
)
if isinstance(model_admin, FailAdmin):
task.delete()
resubmit_task.short_description = _("Resubmit selected tasks to queue")
class TaskAdmin(admin.ModelAdmin):
"""model admin for success tasks."""
list_display = ("name", "group", "func", "started", "stopped", "time_taken")
list_display = (
"name",
"group",
"func",
"cluster",
"started",
"stopped",
"time_taken",
)
actions = [resubmit_task]
def has_add_permission(self, request):
"""Don't allow adds."""
@@ -26,41 +55,47 @@ class TaskAdmin(admin.ModelAdmin):
search_fields = ("name", "func", "group")
readonly_fields = []
list_filter = ("group",)
list_filter = ("group", "cluster")
def get_readonly_fields(self, request, obj=None):
"""Set all fields readonly."""
return list(self.readonly_fields) + [field.name for field in obj._meta.fields]
def retry_failed(FailAdmin, request, queryset):
"""Submit selected tasks back to the queue."""
for task in queryset:
async_task(task.func, *task.args or (), hook=task.hook, **task.kwargs or {})
task.delete()
retry_failed.short_description = _("Resubmit selected tasks to queue")
@admin.display(description=_("Time Taken"))
def time_taken(self, obj):
return obj.time_taken()
class FailAdmin(admin.ModelAdmin):
"""model admin for failed tasks."""
list_display = ("name", "group", "func", "started", "stopped", "short_result")
list_display = (
"name",
"group",
"func",
"cluster",
"started",
"stopped",
"short_result",
)
def has_add_permission(self, request):
"""Don't allow adds."""
return False
actions = [retry_failed]
actions = [resubmit_task]
search_fields = ("name", "func", "group")
list_filter = ("group",)
list_filter = ("group", "cluster")
readonly_fields = []
def get_readonly_fields(self, request, obj=None):
"""Set all fields readonly."""
return list(self.readonly_fields) + [field.name for field in obj._meta.fields]
@admin.display(description=_("Short result"))
def short_result(self, obj):
return truncatechars(obj.result, 100)
class ScheduleAdmin(admin.ModelAdmin):
"""model admin for schedules"""
@@ -82,18 +117,27 @@ class ScheduleAdmin(admin.ModelAdmin):
readonly_fields = ("cron",)
list_filter = ("next_run", "schedule_type", "cluster")
search_fields = ("name", "func",)
search_fields = (
"name",
"func",
)
list_display_links = ("id", "name")
def get_queryset(self, request):
qs = super().get_queryset(request)
task_query = Task.objects.filter(id=OuterRef('task')).values('id', 'name', 'success')
qs = qs.annotate(task_id=Subquery(task_query.values('id')), task_name=Subquery(task_query.values('name')),
task_success=Subquery(task_query.values('success')))
task_query = Task.objects.filter(id=OuterRef("task")).values(
"id", "name", "success"
)
qs = qs.annotate(
task_id=Subquery(task_query.values("id")),
task_name=Subquery(task_query.values("name")),
task_success=Subquery(task_query.values("success")),
)
return qs
def get_success(self, obj):
return obj.task_success
get_success.boolean = True
get_success.short_description = _("success")
@@ -103,8 +147,9 @@ class ScheduleAdmin(admin.ModelAdmin):
url = reverse("admin:django_q_success_change", args=(obj.task_id,))
else:
url = reverse("admin:django_q_failure_change", args=(obj.task_id,))
return format_html(f'<a href="{url}">[{obj.task_name}]</a>')
return format_html('<a href="{}">[{}]</a>', url, obj.task_name)
return None
get_last_run.allow_tags = True
get_last_run.short_description = _("last_run")
@@ -112,7 +157,19 @@ class ScheduleAdmin(admin.ModelAdmin):
class QueueAdmin(admin.ModelAdmin):
"""queue admin for ORM broker"""
list_display = ("id", "key", "name", "group", "func", "lock", "task_id")
list_display = ("id", "key", "name", "group", "func", "lock", "task_id")
fields = (
"key",
"lock",
"task_id",
"name",
"group",
"func",
"args",
"kwargs",
"q_options",
)
readonly_fields = fields[2:]
def save_model(self, request, obj, form, change):
obj.save(using=Conf.ORM)

View File

@@ -9,4 +9,4 @@ class DjangoQConfig(AppConfig):
default_auto_field = "django.db.models.AutoField"
def ready(self):
from django_q.signals import call_hook
from django_q.signals import call_hook # noqa: F401

View File

@@ -7,7 +7,9 @@ from django_q.conf import Conf
class Broker:
def __init__(self, list_key: str = Conf.PREFIX):
def __init__(self, list_key: str = None):
# With same BROKER_CLASS, `list_key` is just a synonym for `queue_name` except for RedisBroker
list_key = list_key or Conf.CLUSTER_NAME
self.connection = self.get_connection(list_key)
self.list_key = list_key
self.cache = self.get_cache()
@@ -104,8 +106,13 @@ class Broker:
return
key_list = self.cache.get(Conf.Q_STAT, [])
if key not in key_list:
# Prune stale entries whose per-stat value has expired, so the
# master list cannot grow without bound across cluster restarts.
key_list = [k for k in key_list if self.cache.get(k) is not None]
key_list.append(key)
self.cache.set(Conf.Q_STAT, key_list)
# timeout=None: master list lifetime is managed by membership
# changes, not by TTL refresh on every heartbeat.
self.cache.set(Conf.Q_STAT, key_list, None)
return self.cache.set(key, value, timeout)
def get_stat(self, key: str):
@@ -151,7 +158,7 @@ class Broker:
return None
@staticmethod
def get_connection(list_key: str = Conf.PREFIX):
def get_connection(list_key: str = None):
"""
Gets a connection to the broker
:param list_key: Optional queue name
@@ -160,13 +167,14 @@ class Broker:
return 0
def get_broker(list_key: str = Conf.PREFIX) -> Broker:
def get_broker(list_key: str = None) -> Broker:
"""
Gets the configured broker type
:param list_key: optional queue name
:type list_key: str
:return: a broker instance
"""
list_key = list_key or Conf.CLUSTER_NAME
# custom
if Conf.BROKER_CLASS:
module, func = Conf.BROKER_CLASS.rsplit(".", 1)
@@ -179,7 +187,7 @@ def get_broker(list_key: str = Conf.PREFIX) -> Broker:
return ironmq.IronMQBroker(list_key=list_key)
# SQS
elif type(Conf.SQS) == dict:
elif isinstance(Conf.SQS, dict):
from django_q.brokers import aws_sqs
return aws_sqs.Sqs(list_key=list_key)

View File

@@ -1,3 +1,5 @@
import copy
from boto3 import Session
from botocore.client import ClientError
@@ -8,7 +10,7 @@ QUEUE_DOES_NOT_EXIST = "AWS.SimpleQueueService.NonExistentQueue"
class Sqs(Broker):
def __init__(self, list_key: str = Conf.PREFIX):
def __init__(self, list_key: str = None):
self.sqs = None
super(Sqs, self).__init__(list_key)
self.queue = self.get_queue()
@@ -39,7 +41,8 @@ class Sqs(Broker):
raise ValueError("receive_message_wait_time_seconds should be int")
if wait_time_second > 20:
raise ValueError(
"receive_message_wait_time_seconds is invalid. Reason: Must be >= 0 and <= 20"
"receive_message_wait_time_seconds is invalid. Reason: Must be >= 0"
" and <= 20"
)
params.update({"WaitTimeSeconds": wait_time_second})
@@ -76,16 +79,16 @@ class Sqs(Broker):
return "AWS SQS"
@staticmethod
def get_connection(list_key: str = Conf.PREFIX) -> Session:
config = Conf.SQS
if "aws_region" in config:
config["region_name"] = config["aws_region"]
del config["aws_region"]
def get_connection(list_key: str = None) -> Session:
config_cloned = copy.deepcopy(Conf.SQS)
if "aws_region" in config_cloned:
config_cloned["region_name"] = config_cloned["aws_region"]
del config_cloned["aws_region"]
if "receive_message_wait_time_seconds" in config:
del config["receive_message_wait_time_seconds"]
if "receive_message_wait_time_seconds" in config_cloned:
del config_cloned["receive_message_wait_time_seconds"]
return Session(**config)
return Session(**config_cloned)
def get_queue(self):
self.sqs = self.connection.resource("sqs")

View File

@@ -46,6 +46,7 @@ class IronMQBroker(Broker):
return self.delete(task_id)
@staticmethod
def get_connection(list_key: str = Conf.PREFIX) -> Queue:
def get_connection(list_key: str = None) -> Queue:
list_key = list_key or Conf.CLUSTER_NAME
ironmq = IronMQ(name=None, **Conf.IRON_MQ)
return ironmq.queue(queue_name=list_key)

View File

@@ -15,7 +15,7 @@ def _timeout():
class Mongo(Broker):
def __init__(self, list_key=Conf.PREFIX):
def __init__(self, list_key: str = None):
super(Mongo, self).__init__(list_key)
self.collection = self.get_collection()
@@ -24,7 +24,7 @@ class Mongo(Broker):
self.collection = self.get_collection()
@staticmethod
def get_connection(list_key: str = Conf.PREFIX) -> MongoClient:
def get_connection(list_key: str = None) -> MongoClient:
return MongoClient(**Conf.MONGO)
def get_collection(self):

View File

@@ -10,13 +10,9 @@ from django_q.conf import Conf, logger
from django_q.models import OrmQ
def _timeout():
return timezone.now() - timedelta(seconds=Conf.RETRY)
class ORM(Broker):
@staticmethod
def get_connection(list_key: str = Conf.PREFIX):
def get_connection(list_key: str = None):
if transaction.get_autocommit(
using=Conf.ORM
): # Only True when not in an atomic block
@@ -28,16 +24,21 @@ class ORM(Broker):
logger.debug("Broker in an atomic transaction")
return OrmQ.objects.using(Conf.ORM)
def timeout(self, task):
return timezone.now() + timedelta(seconds=Conf.RETRY)
def queue_size(self) -> int:
return (
self.get_connection()
.filter(key=self.list_key, lock__lte=_timeout())
.filter(key=self.list_key, lock__lte=timezone.now())
.count()
)
def lock_size(self) -> int:
return (
self.get_connection().filter(key=self.list_key, lock__gt=_timeout()).count()
self.get_connection()
.filter(key=self.list_key, lock__gt=timezone.now())
.count()
)
def purge_queue(self):
@@ -55,14 +56,17 @@ class ORM(Broker):
self.delete(task_id)
def enqueue(self, task):
# list_key might be null (e.g. in a test setup) but OrmQ.key has not-null constraint
package = self.get_connection().create(
key=self.list_key, payload=task, lock=_timeout()
key=self.list_key or Conf.CLUSTER_NAME, payload=task, lock=timezone.now()
)
return package.pk
def dequeue(self):
tasks = self.get_connection().filter(key=self.list_key, lock__lt=_timeout())[
0 : Conf.BULK
tasks = self.get_connection().filter(
key=self.list_key, lock__lt=timezone.now()
)[
0 : Conf.BULK # noqa: E203
]
if tasks:
task_list = []
@@ -70,10 +74,11 @@ class ORM(Broker):
if (
self.get_connection()
.filter(id=task.id, lock=task.lock)
.update(lock=timezone.now())
.update(lock=self.timeout(task))
):
task_list.append((task.pk, task.payload))
# else don't process, as another cluster has been faster than us on that task
# else don't process, as another cluster has been faster than us on
# that task
return task_list
# empty queue, spare the cpu
sleep(Conf.POLL)

View File

@@ -11,7 +11,8 @@ except ImportError:
class Redis(Broker):
def __init__(self, list_key: str = Conf.PREFIX):
def __init__(self, list_key: str = None):
list_key = list_key or Conf.CLUSTER_NAME
super(Redis, self).__init__(list_key=f"django_q:{list_key}:q")
def enqueue(self, task):
@@ -57,7 +58,7 @@ class Redis(Broker):
return self.connection.mget(keys)
@staticmethod
def get_connection(list_key: str = Conf.PREFIX) -> Redis:
def get_connection(list_key: str = None) -> Redis:
if django_redis and Conf.DJANGO_REDIS:
return django_redis.get_redis_connection(Conf.DJANGO_REDIS)
if isinstance(Conf.REDIS, str):

View File

@@ -1,11 +1,8 @@
# Standard
import ast
import pydoc
import os
import signal
import socket
import traceback
import uuid
from datetime import datetime, timedelta
from multiprocessing import Event, Process, Value, current_process
from time import sleep
@@ -20,51 +17,52 @@ except core.exceptions.AppRegistryNotReady:
django.setup()
from django.conf import settings
from django.utils import timezone
from django.utils.translation import gettext_lazy as _
# Local
import django_q.tasks
from django_q.brokers import Broker, get_broker
from django_q.conf import (
Conf,
croniter,
error_reporter,
get_ppid,
logger,
prometheus_multiprocess,
psutil,
resource,
setproctitle,
)
from django_q.humanhash import humanize
from django_q.models import Schedule, Success, Task
from django_q.monitor import monitor
from django_q.pusher import pusher
from django_q.queues import Queue
from django_q.signals import post_execute, pre_execute
from django_q.signing import BadSignature, SignedPackage
from django_q.scheduler import scheduler
from django_q.status import Stat, Status
from .utils import add_months, add_years, get_func_repr
from django_q.worker import worker
class Cluster:
def __init__(self, broker: Broker = None):
self.broker = broker or get_broker()
# Cluster do not need an init or default broker except for testing,
# The sentinel will create a broker for cluster and utilize ALT_CLUSTERS config in Conf.
self.broker = broker # DON'T USE get_broker() to set a default broker here.
self.sentinel = None
self.stop_event = None
self.start_event = None
self.pid = current_process().pid
self.cluster_id = uuid.uuid4()
self.host = socket.gethostname()
self.timeout = Conf.TIMEOUT
self.timeout = None
signal.signal(signal.SIGTERM, self.sig_handler)
signal.signal(signal.SIGINT, self.sig_handler)
def start(self) -> int:
if setproctitle:
setproctitle.setproctitle(f"qcluster {current_process().name} {self.name}")
# Start Sentinel
self.stop_event = Event()
self.start_event = Event()
self.sentinel = Process(
target=Sentinel,
name=f"Process-{uuid.uuid4().hex}",
args=(
self.stop_event,
self.start_event,
@@ -74,7 +72,7 @@ class Cluster:
),
)
self.sentinel.start()
logger.info(_("Q Cluster %(name)s starting.") % {'name': self.name})
logger.info(_("Q Cluster %(name)s starting.") % {"name": self.name})
while not self.start_event.is_set():
sleep(0.1)
return self.pid
@@ -82,19 +80,21 @@ class Cluster:
def stop(self) -> bool:
if not self.sentinel.is_alive():
return False
logger.info(_("Q Cluster %(name)s stopping.") % {'name': self.name})
logger.info(_("Q Cluster %(name)s stopping.") % {"name": self.name})
self.stop_event.set()
self.sentinel.join()
logger.info(_("Q Cluster %(name)s has stopped.") % {'name': self.name})
logger.info(_("Q Cluster %(name)s has stopped.") % {"name": self.name})
self.start_event = None
self.stop_event = None
return True
def sig_handler(self, signum, frame):
logger.debug(
_(
'%(name)s got signal %(signal)s'
) % {'name': current_process().name, 'signal': Conf.SIGNAL_NAMES.get(signum, "UNKNOWN")}
_("%(name)s got signal %(signal)s")
% {
"name": current_process().name,
"signal": Conf.SIGNAL_NAMES.get(signum, "UNKNOWN"),
}
)
self.stop()
@@ -137,7 +137,7 @@ class Sentinel:
start_event,
cluster_id,
broker=None,
timeout=Conf.TIMEOUT,
timeout=None,
start=True,
):
# Make sure we catch signals for the pool
@@ -154,7 +154,7 @@ class Sentinel:
self.start_event = start_event
self.pool_size = Conf.WORKERS
self.pool = []
self.timeout = timeout
self.timeout = timeout or Conf.TIMEOUT
self.task_queue = (
Queue(maxsize=Conf.QUEUE_LIMIT) if Conf.QUEUE_LIMIT else Queue()
)
@@ -165,6 +165,10 @@ class Sentinel:
if start:
self.start()
def queue_name(self):
# multi-queue: cluster name is (broker's) queue_name
return self.broker.list_key if self.broker else "--"
def start(self):
self.broker.ping()
self.spawn_cluster()
@@ -186,7 +190,7 @@ class Sentinel:
"""
:type target: function or class
"""
p = Process(target=target, args=args)
p = Process(target=target, args=args, name=f"Process-{uuid.uuid4().hex}")
p.daemon = True
if target == worker:
p.daemon = Conf.DAEMONIZE_WORKERS
@@ -216,21 +220,59 @@ class Sentinel:
db.connections.close_all()
if process == self.monitor:
self.monitor = self.spawn_monitor()
logger.error(_("reincarnated monitor %(name)s after sudden death") % {'name': process.name})
logger.critical(
_("reincarnated monitor %(name)s after sudden death")
% {"name": process.name}
)
elif process == self.pusher:
self.pusher = self.spawn_pusher()
logger.error(_("reincarnated pusher %(name)s after sudden death") % {'name': process.name})
logger.critical(
_("reincarnated pusher %(name)s after sudden death")
% {"name": process.name}
)
else:
# check if prometheus is proper configurated
prometheus_path = os.getenv(
"PROMETHEUS_MULTIPROC_DIR", os.getenv("prometheus_multiproc_dir")
)
if prometheus_multiprocess and prometheus_path:
prometheus_multiprocess.mark_process_dead(process.pid)
self.pool.remove(process)
self.spawn_worker()
if process.timer.value == 0:
# only need to terminate on timeout, otherwise we risk destabilizing the queues
# only need to terminate on timeout, otherwise we risk destabilizing
# the queues
task_name = ""
if psutil:
try:
process_name = psutil.Process(process.pid).name()
name_splits = process_name.split(" ")
task_name = (
name_splits[3]
if len(name_splits) >= 4 and name_splits[2] == "processing"
else ""
)
except psutil.NoSuchProcess:
pass
process.terminate()
logger.warning(_("reincarnated worker %(name)s after timeout") % {'name': process.name})
if task_name:
msg = _(
"reincarnated worker %(name)s after timeout while processing task %(task_name)s"
) % {"name": process.name, "task_name": task_name}
else:
msg = _("reincarnated worker %(name)s after timeout") % {
"name": process.name
}
logger.critical(msg)
elif int(process.timer.value) == -2:
logger.info(_("recycled worker %(name)s") % {'name': process.name})
logger.info(_("recycled worker %(name)s") % {"name": process.name})
else:
logger.error(_("reincarnated worker %(name)s after death") % {'name': process.name})
logger.critical(
_("reincarnated worker %(name)s after death")
% {"name": process.name}
)
self.reincarnations += 1
@@ -252,13 +294,22 @@ class Sentinel:
def guard(self):
logger.info(
_(
"%(name)s guarding cluster %(cluster_name)s"
) % {'name': current_process().name, 'cluster_name': humanize(self.cluster_id.hex)}
_("%(name)s guarding cluster %(cluster_name)s")
% {
"name": current_process().name,
"cluster_name": humanize(self.cluster_id.hex)
+ f" [{self.queue_name()}]",
}
)
self.start_event.set()
Stat(self).save()
logger.info(_("Q Cluster %(cluster_name)s running.") % {'cluster_name': humanize(self.cluster_id.hex)})
logger.info(
_("Q Cluster %(cluster_name)s running.")
% {
"cluster_name": humanize(self.cluster_id.hex)
+ f" [{self.queue_name()}]"
}
)
counter = 0
cycle = Conf.GUARD_CYCLE # guard loop sleep in seconds
# Guard loop. Runs at least once
@@ -292,7 +343,7 @@ class Sentinel:
def stop(self):
Stat(self).save()
name = current_process().name
logger.info(_("%(name)s stopping cluster processes") % {'name': name})
logger.info(_("%(name)s stopping cluster processes") % {"name": name})
# Stopping pusher
self.event_out.set()
# Wait for it to stop
@@ -317,7 +368,7 @@ class Sentinel:
self.result_queue.close()
# Wait for the result queue to empty
self.result_queue.join_thread()
logger.info(_("%(name)s waiting for the monitor.") % {'name': name})
logger.info(_("%(name)s waiting for the monitor.") % {"name": name})
# Wait for everything to close or time out
count = 0
if not self.timeout:
@@ -330,381 +381,6 @@ class Sentinel:
Stat(self).save()
def pusher(task_queue: Queue, event: Event, broker: Broker = None):
"""
Pulls tasks of the broker and puts them in the task queue
:type broker:
:type task_queue: multiprocessing.Queue
:type event: multiprocessing.Event
"""
if not broker:
broker = get_broker()
logger.info(_("%(process_name)s pushing tasks at %(id)s") % {'process_name': current_process().name, 'id': current_process().pid})
while True:
try:
task_set = broker.dequeue()
except Exception as e:
logger.error(e, traceback.format_exc())
# broker probably crashed. Let the sentinel handle it.
sleep(10)
break
if task_set:
for task in task_set:
ack_id = task[0]
# unpack the task
try:
task = SignedPackage.loads(task[1])
except (TypeError, BadSignature) as e:
logger.error(e, traceback.format_exc())
broker.fail(ack_id)
continue
task["ack_id"] = ack_id
task_queue.put(task)
logger.debug(_("queueing from %(list_key)s") % {'list_key': broker.list_key})
if event.is_set():
break
logger.info(_("%(name)s stopped pushing tasks") % {'name': current_process().name})
def monitor(result_queue: Queue, broker: Broker = None):
"""
Gets finished tasks from the result queue and saves them to Django
:type broker: brokers.Broker
:type result_queue: multiprocessing.Queue
"""
if not broker:
broker = get_broker()
name = current_process().name
logger.info(_("%(name)s monitoring at %(id)s") % {'name': name, 'id': current_process().pid})
for task in iter(result_queue.get, "STOP"):
# save the result
if task.get("cached", False):
save_cached(task, broker)
else:
save_task(task, broker)
# acknowledge result
ack_id = task.pop("ack_id", False)
if ack_id and (task["success"] or task.get("ack_failure", False)):
broker.acknowledge(ack_id)
# signal execution done
post_execute.send(sender="django_q", task=task)
# log the result
info_name = get_func_repr(task['func'])
if task["success"]:
# log success
logger.info(_("Processed '%(info_name)s' (%(task_name)s)") % {'info_name': info_name, 'task_name': task['name']})
else:
# log failure
logger.error(_("Failed '%(info_name)s' (%(task_name)s) - %(task_result)s") % {'info_name': info_name, 'task_name': task['name'], 'task_result': task['result']})
logger.info(_("%(name)s stopped monitoring results") % {'name': name})
def worker(
task_queue: Queue, result_queue: Queue, timer: Value, timeout: int = Conf.TIMEOUT
):
"""
Takes a task from the task queue, tries to execute it and puts the result back in the result queue
:param timeout: number of seconds wait for a worker to finish.
:type task_queue: multiprocessing.Queue
:type result_queue: multiprocessing.Queue
:type timer: multiprocessing.Value
"""
proc_name = current_process().name
logger.info(_("%(proc_name)s ready for work at %(id)s") % {'proc_name': proc_name, 'id': current_process().pid})
task_count = 0
if timeout is None:
timeout = -1
# Start reading the task queue
for task in iter(task_queue.get, "STOP"):
result = None
timer.value = -1 # Idle
task_count += 1
# Get the function from the task
func = task["func"]
func_name = get_func_repr(func)
logger.info(_("%(proc_name)s processing '%(func_name)s' (%(task_name)s)") % {'proc_name': proc_name, 'func_name': func_name, 'task_name': task['name']})
f = task["func"]
# if it's not an instance try to get it from the string
if not callable(task["func"]):
f = pydoc.locate(f)
close_old_django_connections()
timer_value = task.pop("timeout", timeout)
# signal execution
pre_execute.send(sender="django_q", func=f, task=task)
# execute the payload
timer.value = timer_value # Busy
try:
res = f(*task["args"], **task["kwargs"])
result = (res, True)
except Exception:
result = (_("Could not process '%(func_name)s'. Check the location of the function and the args/kwargs.") % {'func_name': func_name}, False)
if error_reporter:
error_reporter.report()
if task.get("sync", False):
raise Exception(result)
with timer.get_lock():
# Process result
task["result"] = result[0]
task["success"] = result[1]
task["stopped"] = timezone.now()
result_queue.put(task)
timer.value = -1 # Idle
# Recycle
if task_count == Conf.RECYCLE or rss_check():
timer.value = -2 # Recycled
break
logger.info(_("%(proc_name)s stopped doing work") % {'proc_name': proc_name})
def save_task(task, broker: Broker):
"""
Saves the task package to Django or the cache
:param task: the task package
:type broker: brokers.Broker
"""
# SAVE LIMIT < 0 : Don't save success
if not task.get("save", Conf.SAVE_LIMIT >= 0) and task["success"]:
return
# enqueues next in a chain
if task.get("chain", None):
django_q.tasks.async_chain(
task["chain"],
group=task["group"],
cached=task["cached"],
sync=task["sync"],
broker=broker,
)
# SAVE LIMIT > 0: Prune database, SAVE_LIMIT 0: No pruning
close_old_django_connections()
try:
filters = {}
if Conf.SAVE_LIMIT_PER and Conf.SAVE_LIMIT_PER in {"group", "name", "func"} and Conf.SAVE_LIMIT_PER in task:
value = task[Conf.SAVE_LIMIT_PER]
if Conf.SAVE_LIMIT_PER == "func":
value = get_func_repr(value)
filters[Conf.SAVE_LIMIT_PER] = value
database_to_use = {"using": Conf.ORM if Conf.ORM else Schedule.objects.db} if not Conf.HAS_REPLICA else {}
with db.transaction.atomic(**database_to_use):
last = Success.objects.filter(**filters).select_for_update().last()
if task["success"] and 0 < Conf.SAVE_LIMIT <= Success.objects.filter(**filters).count():
last.delete()
# check if this task has previous results
try:
existing_task = Task.objects.get(id=task["id"], name=task["name"])
# only update the result if it hasn't succeeded yet
if not existing_task.success:
existing_task.stopped = task["stopped"]
existing_task.result = task["result"]
existing_task.success = task["success"]
existing_task.attempt_count = existing_task.attempt_count + 1
existing_task.save()
if (
Conf.MAX_ATTEMPTS > 0
and existing_task.attempt_count >= Conf.MAX_ATTEMPTS
):
broker.acknowledge(task["ack_id"])
except Task.DoesNotExist:
# convert func to string
func = get_func_repr(task["func"])
Task.objects.create(
id=task["id"],
name=task["name"],
func=func,
hook=task.get("hook"),
args=task["args"],
kwargs=task["kwargs"],
started=task["started"],
stopped=task["stopped"],
result=task["result"],
group=task.get("group"),
success=task["success"],
attempt_count=1,
)
except Exception as e:
logger.error(e)
def save_cached(task, broker: Broker):
task_key = f'{broker.list_key}:{task["id"]}'
timeout = task["cached"]
if timeout is True:
timeout = None
try:
group = task.get("group", None)
iter_count = task.get("iter_count", 0)
# if it's a group append to the group list
if group:
group_key = f"{broker.list_key}:{group}:keys"
group_list = broker.cache.get(group_key) or []
# if it's an iter group, check if we are ready
if iter_count and len(group_list) == iter_count - 1:
group_args = f"{broker.list_key}:{group}:args"
# collate the results into a Task result
results = [
SignedPackage.loads(broker.cache.get(k))["result"]
for k in group_list
]
results.append(task["result"])
task["result"] = results
task["id"] = group
task["args"] = SignedPackage.loads(broker.cache.get(group_args))
task.pop("iter_count", None)
task.pop("group", None)
if task.get("iter_cached", None):
task["cached"] = task.pop("iter_cached", None)
save_cached(task, broker=broker)
else:
save_task(task, broker)
broker.cache.delete_many(group_list)
broker.cache.delete_many([group_key, group_args])
return
# save the group list
group_list.append(task_key)
broker.cache.set(group_key, group_list, timeout)
# async_task next in a chain
if task.get("chain", None):
django_q.tasks.async_chain(
task["chain"],
group=group,
cached=task["cached"],
sync=task["sync"],
broker=broker,
)
# save the task
broker.cache.set(task_key, SignedPackage.dumps(task), timeout)
except Exception as e:
logger.error(e)
def scheduler(broker: Broker = None):
"""
Creates a task from a schedule at the scheduled time and schedules next run
"""
if not broker:
broker = get_broker()
close_old_django_connections()
try:
database_to_use = {"using": Conf.ORM if Conf.ORM else Schedule.objects.db} if not Conf.HAS_REPLICA else {}
with db.transaction.atomic(**database_to_use):
for s in (
Schedule.objects.select_for_update()
.exclude(repeats=0)
.filter(next_run__lt=timezone.now())
.filter(
db.models.Q(cluster__isnull=True) | db.models.Q(cluster=Conf.PREFIX)
)
):
args = ()
kwargs = {}
# get args, kwargs and hook
if s.kwargs:
try:
# first try the dict syntax
kwargs = ast.literal_eval(s.kwargs)
except (SyntaxError, ValueError):
# else use the kwargs syntax
try:
parsed_kwargs = ast.parse(f"f({s.kwargs})").body[0].value.keywords
kwargs = {kwarg.arg: ast.literal_eval(kwarg.value) for kwarg in parsed_kwargs}
except (SyntaxError, ValueError):
kwargs = {}
if s.args:
args = ast.literal_eval(s.args)
# single value won't eval to tuple, so:
if type(args) != tuple:
args = (args,)
q_options = kwargs.get("q_options", {})
if s.hook:
q_options["hook"] = s.hook
# set up the next run time
if s.schedule_type != s.ONCE:
next_run = s.next_run
while True:
if s.schedule_type == s.MINUTES:
next_run = next_run + timedelta(minutes=(s.minutes or 1))
elif s.schedule_type == s.HOURLY:
next_run = next_run + timedelta(hours=1)
elif s.schedule_type == s.DAILY:
next_run = next_run + timedelta(days=1)
elif s.schedule_type == s.WEEKLY:
next_run = next_run + timedelta(weeks=1)
elif s.schedule_type == s.BIWEEKLY:
next_run = next_run + timedelta(weeks=2)
elif s.schedule_type == s.MONTHLY:
next_run = add_months(next_run, 1)
elif s.schedule_type == s.BIMONTHLY:
next_run = add_months(next_run, 2)
elif s.schedule_type == s.QUARTERLY:
next_run = add_months(next_run, 3)
elif s.schedule_type == s.YEARLY:
next_run = add_years(next_run, 1)
elif s.schedule_type == s.CRON:
if not croniter:
raise ImportError(
_(
"Please install croniter to enable cron expressions"
)
)
next_run = croniter(s.cron, localtime()).get_next(datetime)
if Conf.CATCH_UP or next_run > localtime():
break
s.next_run = next_run
s.repeats += -1
# send it to the cluster
scheduled_broker = broker
try:
scheduled_broker = get_broker(q_options["broker_name"])
except: # invalid broker_name or non existing broker with broker_name
pass
q_options["broker"] = scheduled_broker
q_options["group"] = q_options.get("group", s.name or s.id)
kwargs["q_options"] = q_options
s.task = django_q.tasks.async_task(s.func, *args, **kwargs)
# log it
if not s.task:
logger.error(
_(
"%(process_name)s failed to create a task from schedule [%(schedule)s]"
) % {'process_name': current_process().name, 'schedule': s.name or s.id}
)
else:
logger.info(
_(
"%(process_name)s created a task from schedule [%(schedule)s]"
) % {'process_name': current_process().name, 'schedule': s.name or s.id}
)
# default behavior is to delete a ONCE schedule
if s.schedule_type == s.ONCE:
if s.repeats < 0:
s.delete()
continue
# but not if it has a positive repeats
s.repeats = 0
# save the schedule
s.save()
except Exception as e:
logger.error(e)
def close_old_django_connections():
"""
Close django connections unless running with sync=True.
"""
if Conf.SYNC:
logger.warning(
"Preserving django database connections because sync=True. Beware "
"that tasks are now injected in the calling context/transactions "
"which may result in unexpected behaviour."
)
else:
db.close_old_connections()
def set_cpu_affinity(n: int, process_ids: list, actual: bool = not Conf.TESTING):
"""
Sets the cpu affinity for the supplied processes.
@@ -741,20 +417,7 @@ def set_cpu_affinity(n: int, process_ids: list, actual: bool = not Conf.TESTING)
p = psutil.Process(pid)
if actual:
p.cpu_affinity(affinity)
logger.info(_("%(pid)s will use cpu %(affinity)s") % {'pid': pid, 'affinity': affinity})
def rss_check():
if Conf.MAX_RSS:
if resource:
return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss >= Conf.MAX_RSS
elif psutil:
return psutil.Process().memory_info().rss >= Conf.MAX_RSS * 1024
return False
def localtime() -> datetime:
"""Override for timezone.localtime to deal with naive times and local times"""
if settings.USE_TZ:
return timezone.localtime()
return datetime.now()
logger.info(
_("%(pid)s will use cpu %(affinity)s")
% {"pid": pid, "affinity": affinity}
)

View File

@@ -1,16 +1,22 @@
import logging
import os
import signal
import sys
from copy import deepcopy
from multiprocessing import cpu_count
from signal import signal
from warnings import warn
import pkg_resources
from django.conf import settings
from django.utils.translation import gettext_lazy as _
from django_q.queues import Queue
# The "selectable" entry points were introduced in importlib_metadata 3.6 and Python 3.10.
if sys.version_info < (3, 10):
from importlib_metadata import entry_points
else:
from importlib.metadata import entry_points
# optional
try:
import psutil
@@ -27,6 +33,16 @@ try:
except ModuleNotFoundError:
resource = None
try:
import setproctitle
except ModuleNotFoundError:
setproctitle = None
try:
from prometheus_client import multiprocess as prometheus_multiprocess
except ModuleNotFoundError:
prometheus_multiprocess = None
class Conf:
"""
@@ -34,10 +50,25 @@ class Conf:
"""
try:
conf = settings.Q_CLUSTER
conf = settings.Q_CLUSTER.copy()
except AttributeError:
conf = {}
_Q_CLUSTER_NAME = os.getenv("Q_CLUSTER_NAME")
if (
_Q_CLUSTER_NAME
and _Q_CLUSTER_NAME != conf.get("name")
and _Q_CLUSTER_NAME != conf.get("cluster_name")
):
conf["cluster_name"] = _Q_CLUSTER_NAME
alt_conf = conf.pop("ALT_CLUSTERS")
if isinstance(alt_conf, dict):
alt_conf = alt_conf.get(_Q_CLUSTER_NAME)
if isinstance(alt_conf, dict):
alt_conf.pop("name", None)
alt_conf.pop("cluster_name", None)
conf.update(alt_conf)
# Redis server configuration . Follows standard redis keywords
REDIS = conf.get("redis", {})
@@ -54,9 +85,6 @@ class Conf:
# ORM broker
ORM = conf.get("orm", None)
# ORM support for read/write replicas
HAS_REPLICA = conf.get("has_replica", False)
# Custom broker class
BROKER_CLASS = conf.get("broker_class", None)
@@ -68,12 +96,19 @@ class Conf:
MONGO_DB = conf.get("mongo_db", None)
# Name of the cluster or site. For when you run multiple sites on one redis server
# It's also the `salt` for signing OrmQ, and part of the Redis stats caching key
# For all clusters in one site, PREFIX should be the same value to be able to decrypt payloads
PREFIX = conf.get("name", "default")
# Support alternative cluster name to use multiple queues in one site.
# cluster name and queue name are interchangeable, same thing.
CLUSTER_NAME = conf.get("cluster_name", PREFIX)
# Log output level
LOG_LEVEL = conf.get("log_level", "INFO")
# Maximum number of successful tasks kept in the database. 0 saves everything. -1 saves none
# Maximum number of successful tasks kept in the database. 0 saves everything.
# -1 saves none
# Failures are always saved
SAVE_LIMIT = conf.get("save_limit", 250)
@@ -82,7 +117,13 @@ class Conf:
# Verify SAVE_LIMIT_PER is valid
if SAVE_LIMIT_PER not in ["group", "name", "func", None]:
warn(_("SAVE_LIMIT_PER (%(option)s) is not a valid option. Options are: 'group', 'name', 'func' and None. Default is None.") % {'option': SAVE_LIMIT_PER})
warn(
_(
"SAVE_LIMIT_PER (%(option)s) is not a valid option. Options are: "
"'group', 'name', 'func' and None. Default is None."
)
% {"option": SAVE_LIMIT_PER}
)
# Guard loop sleep in seconds. Should be between 0 and 60 seconds.
GUARD_CYCLE = conf.get("guard_cycle", 0.5)
@@ -113,11 +154,12 @@ class Conf:
# Sets compression of redis packages
COMPRESSED = conf.get("compress", False)
# Number of tasks each worker can handle before it gets recycled. Useful for releasing memory
# Number of tasks each worker can handle before it gets recycled.
# Useful for releasing memory
RECYCLE = conf.get("recycle", 500)
# The maximum resident set size in kilobytes before a worker will recycle. Useful for limiting memory usage
# Not available on all platforms
# The maximum resident set size in kilobytes before a worker will recycle.
# Useful for limiting memory usage. Not available on all platforms
MAX_RSS = conf.get("max_rss", None)
# Number of seconds to wait for a worker to finish.
@@ -135,9 +177,10 @@ class Conf:
# Verify if retry and timeout settings are correct
if not TIMEOUT or (TIMEOUT > RETRY):
warn(
"""Retry and timeout are misconfigured. Set retry larger than timeout,
failure to do so will cause the tasks to be retriggered before completion.
See https://django-q2.readthedocs.io/en/master/configure.html#retry for details."""
"Retry and timeout are misconfigured. Set retry larger than timeout,"
"failure to do so will cause the tasks to be retriggered before completion."
"See https://django-q2.readthedocs.io/en/master/configure.html#retry "
"for details."
)
# Sets the amount of tasks the cluster will try to pop off the broker.
@@ -156,12 +199,14 @@ class Conf:
# The Django cache to use
CACHE = conf.get("cache", "default")
# Use the cache as result backend. Can be 'True' or an integer representing the global cache timeout.
# Use the cache as result backend. Can be 'True' or an integer representing the
# global cache timeout.
# i.e 'cached: 60' , will make all results go the cache and expire in 60 seconds.
CACHED = conf.get("cached", False)
# If set to False the scheduler won't execute tasks in the past.
# Instead it will run once and reschedule the next run in the future. Defaults to True.
# Instead it will run once and reschedule the next run in the future. Defaults to
# True.
CATCH_UP = conf.get("catch_up", True)
# Use the secret key for package signing
@@ -200,6 +245,11 @@ class Conf:
# to manage workarounds during testing
TESTING = conf.get("testing", False)
# Timezone for next_run, overrules Django timezone
TIME_ZONE = None
if settings.USE_TZ:
TIME_ZONE = conf.get("time_zone", settings.TIME_ZONE)
# logger
logger = logging.getLogger("django-q")
@@ -218,7 +268,6 @@ if not logger.hasHandlers():
# Error Reporting Interface
class ErrorReporter:
# initialize with iterator of reporters (better name, targets?)
def __init__(self, reporters):
self.targets = [target for target in reporters]
@@ -237,9 +286,7 @@ if Conf.ERROR_REPORTER:
# iterate through the configured error reporters,
# and instantiate an ErrorReporter using the provided config
for name, conf in error_conf.items():
for entry in pkg_resources.iter_entry_points(
"djangoq.errorreporters", name
):
for entry in entry_points(group="djangoq.errorreporters", name=name):
Reporter = entry.load()
reporters.append(Reporter(**conf))
error_reporter = ErrorReporter(reporters)
@@ -257,5 +304,6 @@ def get_ppid():
return psutil.Process(os.getpid()).ppid()
else:
raise OSError(
"Your OS does not support `os.getppid`. Please install `psutil` as an alternative provider."
"Your OS does not support `os.getppid`. Please install `psutil` as an "
"alternative provider."
)

View File

@@ -2,15 +2,24 @@ import datetime
import time
import zlib
from django.core.signing import BadSignature, JSONSerializer, SignatureExpired
from django.core.signing import (
BadSignature,
JSONSerializer,
SignatureExpired,
b64_decode,
dumps,
)
from django.core.signing import Signer as Sgnr
from django.core.signing import TimestampSigner as TsS
from django.core.signing import b64_decode, dumps
try:
from django.core.signing import base62
from django.core.signing import b62_decode
except ImportError:
# For django < 4.0
# fallback for django 3.x
from django.utils.baseconv import base62
b62_decode = base62.decode
from django.utils.crypto import constant_time_compare
from django.utils.encoding import force_bytes, force_str
@@ -36,7 +45,9 @@ def loads(
"""
# TimestampSigner.unsign() returns str but base64 and zlib compression
# operate on bytes.
base64d = force_bytes(TimestampSigner(key, salt=salt).unsign(s, max_age=max_age))
base64d = force_bytes(
TimestampSigner(key=key, salt=salt).unsign(s, max_age=max_age)
)
decompress = False
if base64d[:1] == b".":
# It's compressed; uncompress it first
@@ -73,7 +84,7 @@ class TimestampSigner(Signer, TsS):
"""
result = super(TimestampSigner, self).unsign(value)
value, timestamp = result.rsplit(self.sep, 1)
timestamp = base62.decode(timestamp)
timestamp = b62_decode(timestamp)
if max_age is not None:
if isinstance(max_age, datetime.timedelta):
max_age = max_age.total_seconds()

7
django_q/exceptions.py Normal file
View File

@@ -0,0 +1,7 @@
class TimeoutException(SystemExit):
"""
Exception for when a worker takes too long to complete a task
Raising SystemExit will make sure the function terminates gracefully.
"""
pass

View File

@@ -4,6 +4,7 @@ humanhash: Human-readable representations of digests.
The simplest ways to use this module are the :func:`humanize` and :func:`uuid`
functions. For tighter control over the output, see :class:`HumanHasher`.
"""
import operator
import uuid as uuidlib
from argparse import ArgumentError
@@ -270,7 +271,6 @@ DEFAULT_WORDLIST = (
class HumanHasher:
"""
Transforms hex digests to human-readable strings.
@@ -290,7 +290,6 @@ class HumanHasher:
self.wordlist = wordlist
def humanize(self, hexdigest, words=4, separator="-"):
"""
Humanize a given hexadecimal digest.
@@ -314,7 +313,6 @@ class HumanHasher:
@staticmethod
def compress(bytes, target):
"""
Compress a list of byte values to a fixed target length.
@@ -337,17 +335,22 @@ class HumanHasher:
# Split `bytes` into `target` segments.
seg_size = length // target
segments = [bytes[i * seg_size : (i + 1) * seg_size] for i in range(target)]
# fmt: off
segments = [
bytes[i * seg_size : (i + 1) * seg_size] for i in range(target) # noqa: E203 E501
]
# fmt: on
# Catch any left-over bytes in the last segment.
segments[-1].extend(bytes[target * seg_size :])
segments[-1].extend(bytes[target * seg_size :]) # noqa: E203 E501
# Use a simple XOR checksum-like function for compression.
checksum = lambda bytes: reduce(operator.xor, bytes, 0)
def checksum(bytes):
return reduce(operator.xor, bytes, 0)
checksums = list(map(checksum, segments))
return checksums
def uuid(self, **params):
"""
Generate a UUID with a human-readable representation.

View File

@@ -6,7 +6,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2022-11-12 01:47+0000\n"
"POT-Creation-Date: 2026-04-17 21:10+0000\n"
"PO-Revision-Date: 2018-08-05 18:28+0200\n"
"Last-Translator: Jonas Winkler\n"
"Language-Team: \n"
@@ -17,177 +17,118 @@ msgstr ""
"X-Generator: Poedit 2.1.1\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"
#: django_q/admin.py:43
#: admin.py:30
msgid "Resubmit selected tasks to queue"
msgstr "Ausgewählte Aufgaben erneut ausführen"
#: django_q/admin.py:98 django_q/models.py:228
#: admin.py:64
msgid "Time Taken"
msgstr ""
#: admin.py:95
msgid "Short result"
msgstr ""
#: admin.py:142 models.py:412
#, fuzzy
#| msgid "Success"
msgid "success"
msgstr "erfolg"
#: django_q/admin.py:109 django_q/models.py:230
#: admin.py:154 models.py:414
msgid "last_run"
msgstr ""
#: django_q/cluster.py:77
#: cluster.py:75
#, python-format
msgid "Q Cluster %(name)s starting."
msgstr "Q-Cluster %(name)s wird gestartet."
#: django_q/cluster.py:85
#: cluster.py:83
#, fuzzy, python-format
#| msgid "Q Cluster-{} stopping."
msgid "Q Cluster %(name)s stopping."
msgstr "Q-Cluster {name} wird gestoppt."
#: django_q/cluster.py:88
#: cluster.py:86
#, python-format
msgid "Q Cluster %(name)s has stopped."
msgstr "Q-Cluster %(name)s wurde gestoppt."
#: django_q/cluster.py:96
#: cluster.py:93
#, python-format
msgid "%(name)s got signal %(signal)s"
msgstr "%(name)s erhielt das Signal %(signal)s"
#: django_q/cluster.py:219
#: cluster.py:224
#, python-format
msgid "reincarnated monitor %(name)s after sudden death"
msgstr "Monitor %(name)s wurde nach unerwartetem Absturz neu gestartet"
#: django_q/cluster.py:222
#: cluster.py:230
#, python-format
msgid "reincarnated pusher %(name)s after sudden death"
msgstr "Pusher %(name)s wurde nach unerwartetem Absturz neu gestartet"
#: django_q/cluster.py:229
#: cluster.py:262
#, fuzzy, python-format
#| msgid "reincarnated worker %(name)s after timeout"
msgid ""
"reincarnated worker %(name)s after timeout while processing task "
"%(task_name)s"
msgstr "Worker %(name)s wurde nach Zeitüberschreitung neu gestartet"
#: cluster.py:265
#, python-format
msgid "reincarnated worker %(name)s after timeout"
msgstr "Worker %(name)s wurde nach Zeitüberschreitung neu gestartet"
#: django_q/cluster.py:231
#: cluster.py:270
#, python-format
msgid "recycled worker %(name)s"
msgstr "Worker %(name)s wurde wiederverwendet"
#: django_q/cluster.py:233
#: cluster.py:273
#, python-format
msgid "reincarnated worker %(name)s after death"
msgstr "Worker %(name)s wurde nach unerwartetem Absturz neu gestartet"
#: django_q/cluster.py:256
#: cluster.py:297
#, python-format
msgid "%(name)s guarding cluster %(cluster_name)s"
msgstr "%(name)s beschützt das Cluster %(cluster_name)s"
#: django_q/cluster.py:261
#: cluster.py:307
#, python-format
msgid "Q Cluster %(cluster_name)s running."
msgstr "Q-Cluster %(cluster_name)s läuft."
#: django_q/cluster.py:295
#: cluster.py:346
#, python-format
msgid "%(name)s stopping cluster processes"
msgstr "%(name)s hält Cluster-Prozesse an"
#: django_q/cluster.py:320
#: cluster.py:371
#, python-format
msgid "%(name)s waiting for the monitor."
msgstr "%(name)s wartet auf den Monitor."
#: django_q/cluster.py:342
#, python-format
msgid "%(process_name)s pushing tasks at %(id)s"
msgstr "%(process_name)s veröffentlicht Aufagaben auf %(id)s"
#: django_q/cluster.py:363
#, python-format
msgid "queueing from %(list_key)s"
msgstr "Einreihen von %(list_key)s"
#: django_q/cluster.py:366
#, python-format
msgid "%(name)s stopped pushing tasks"
msgstr "%(name)s veröffentlicht keine Aufgaben mehr"
#: django_q/cluster.py:378
#, python-format
msgid "%(name)s monitoring at %(id)s"
msgstr "%(name)s beobachtet auf %(id)s"
#: django_q/cluster.py:395
#, python-format
msgid "Processed '%(info_name)s' (%(task_name)s)"
msgstr "[%(task_name)s] - '%(info_name)s' wurde verarbeitet"
#: django_q/cluster.py:398
#, python-format
msgid "Failed '%(info_name)s' (%(task_name)s) - %(task_result)s"
msgstr "'%(info_name)s' (%(task_name)s) ist fehlgeschlagen - %(task_result)s"
#: django_q/cluster.py:399
#, python-format
msgid "%(name)s stopped monitoring results"
msgstr "%(name)s überwacht keine Ergebnisse mehr"
#: django_q/cluster.py:413
#, python-format
msgid "%(proc_name)s ready for work at %(id)s"
msgstr "%(proc_name)s ist bereit für Arbeit auf %(id)s"
#: django_q/cluster.py:425
#, python-format
msgid "%(proc_name)s processing '%(func_name)s' (%(task_name)s)"
msgstr "%(proc_name)s verarbeitet '%(func_name)s' (%(task_name)s)"
#: django_q/cluster.py:440
#, python-format
msgid ""
"Could not process '%(func_name)s'. Check the location of the function and "
"the args/kwargs."
msgstr ""
"Konnte '%(func_name)s' nicht verarbeiten. Überprüfen Sie den Ort der "
"Funktion und die args/kwargs."
#: django_q/cluster.py:456
#, python-format
msgid "%(proc_name)s stopped doing work"
msgstr "%(proc_name)s hat die Arbeit beendet"
#: django_q/cluster.py:649 django_q/models.py:144
msgid "Please install croniter to enable cron expressions"
msgstr "Bitte installieren Sie croniter, um Cron-Ausdrücke zu aktivieren"
#: django_q/cluster.py:672
#, python-format
msgid "%(process_name)s failed to create a task from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s konnte keine Aufgabe von Zeitplan [%(schedule)s] erstellen"
#: django_q/cluster.py:678
#, python-format
msgid "%(process_name)s created a task from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s hat eine Aufgabe des Zeitplans [%(schedule)s] erstellt"
#: django_q/cluster.py:718
#: cluster.py:394
msgid "Skipping cpu affinity because psutil was not found."
msgstr "Cpu-Affinität wird übersprungen, da psutil nicht gefunden wurde."
#: django_q/cluster.py:723
#: cluster.py:399
msgid "Faking cpu affinity because it is not supported on this platform"
msgstr ""
"Vortäuschen von CPU-Affinität, da diese auf dieser Plattform nicht "
"unterstützt wird"
#: django_q/cluster.py:744
#: cluster.py:421
#, python-format
msgid "%(pid)s will use cpu %(affinity)s"
msgstr "%(pid)s wird CPU %(affinity)s benutzen"
#: django_q/conf.py:85
#: conf.py:122
#, python-format
msgid ""
"SAVE_LIMIT_PER (%(option)s) is not a valid option. Options are: 'group', "
@@ -197,291 +138,553 @@ msgstr ""
"'group', 'name', 'func' und None. Standard ist None."
#. Translators: Cluster status descriptions
#: django_q/conf.py:194
#: conf.py:239
msgid "Starting"
msgstr "Wird gestartet"
#: django_q/conf.py:195
#: conf.py:240
msgid "Working"
msgstr "Arbeitet"
#: django_q/conf.py:196
#: conf.py:241
msgid "Idle"
msgstr "Leerlauf"
#: django_q/conf.py:197
#: conf.py:242
msgid "Stopped"
msgstr "Gestoppt"
#: django_q/conf.py:198
#: conf.py:243
msgid "Stopping"
msgstr "Wird gestoppt"
#. Translators: help text for qcluster management command
#: django_q/management/commands/qcluster.py:9
#: management/commands/qcluster.py:11
msgid "Starts a Django Q Cluster."
msgstr "Startet ein Django-Q-Cluster."
#. Translators: help text for qinfo management command
#: django_q/management/commands/qinfo.py:11
#: management/commands/qinfo.py:11
msgid "General information over all clusters."
msgstr "Allgemeine Informationen über alle Cluster"
#. Translators: help text for qmemory management command
#: django_q/management/commands/qmemory.py:9
#: management/commands/qmemory.py:9
msgid "Monitors Q Cluster memory usage"
msgstr "Überwacht die Speichernutzung von Q Cluster"
#. Translators: help text for qmonitor management command
#: django_q/management/commands/qmonitor.py:9
#: management/commands/qmonitor.py:9
msgid "Monitors Q Cluster activity"
msgstr "Q-Cluster aktiv überwachen"
#: django_q/models.py:119
msgid "Successful task"
msgstr "Erfolgreiche Aufgabe"
#: models.py:30 models.py:469
msgid "Task id"
msgstr ""
#: django_q/models.py:120
msgid "Successful tasks"
msgstr "Erfolgreiche Aufgaben"
#: models.py:31
msgid "32-character task identifier."
msgstr ""
#: django_q/models.py:135
msgid "Failed task"
msgstr "Fehlgeschlagene Aufgabe"
#: models.py:36 models.py:235 models.py:470
msgid "Name"
msgstr ""
#: django_q/models.py:136
msgid "Failed tasks"
msgstr "Fehlgeschlagene Aufgaben"
#: models.py:37
msgid "Optional human-readable name for lookup and display."
msgstr ""
#: django_q/models.py:160
msgid "e.g. 1, 2, 'John'"
msgstr "zum Beispiel 1, 2, 'John'"
#: models.py:41 models.py:240 models.py:468
msgid "Function"
msgstr ""
#: django_q/models.py:162
msgid "e.g. x=1, y=2, name='John'"
msgstr "zum Beispiel x=1, y=2, name='John'"
#: models.py:42
msgid "Dotted import path to the callable, e.g. myapp.tasks.job."
msgstr ""
#: django_q/models.py:176
msgid "Once"
msgstr "Einmal"
#: models.py:47 models.py:247
msgid "Hook"
msgstr ""
#: django_q/models.py:177
msgid "Minutes"
msgstr "Minuten"
#: models.py:48
msgid "Optional dotted path to a callback that receives the task result."
msgstr ""
#: django_q/models.py:178
msgid "Hourly"
msgstr "Stündlich"
#: models.py:53 models.py:253 models.py:472
msgid "Arguments"
msgstr ""
#: django_q/models.py:179
msgid "Daily"
msgstr "Täglich"
#: models.py:54
msgid "Pickled positional arguments (tuple)."
msgstr ""
#: django_q/models.py:180
msgid "Weekly"
msgstr "Wöchentlich"
#: models.py:59 models.py:259 models.py:473
msgid "Keyword arguments"
msgstr ""
#: django_q/models.py:181
msgid "Biweekly"
msgstr "Zweiwöchentlich"
#: models.py:60
msgid "Pickled keyword arguments (dict)."
msgstr ""
#: django_q/models.py:182
msgid "Monthly"
msgstr "Monatlich"
#: models.py:65
msgid "Result"
msgstr ""
#: django_q/models.py:183
msgid "Bimonthly"
msgstr "Zweimonatlich"
#: models.py:66
msgid "Pickled return value or error payload."
msgstr ""
#: django_q/models.py:184
msgid "Quarterly"
msgstr "Vierteljährlich"
#: models.py:72 models.py:471
msgid "Group"
msgstr ""
#: django_q/models.py:185
msgid "Yearly"
msgstr "Jährlich"
#: models.py:73
msgid "Optional group id to aggregate related tasks."
msgstr ""
#: django_q/models.py:186
msgid "Cron"
msgstr "Cron"
#: models.py:80 models.py:328
#, fuzzy
#| msgid "Clusters"
msgid "Cluster"
msgstr "Cluster"
#: django_q/models.py:189
msgid "Schedule Type"
msgstr "Zeitplan-Typ"
#: models.py:81
msgid "Name of the cluster that executed this task."
msgstr ""
#: django_q/models.py:192
msgid "Number of minutes for the Minutes type"
msgstr "Anzahl Minuten für den Typ 'Minuten'"
#: models.py:85
msgid "Started at"
msgstr ""
#: django_q/models.py:195
msgid "Repeats"
msgstr "Wiederhohlungen"
#: models.py:86
msgid "When the worker started executing the task."
msgstr ""
#: django_q/models.py:195
msgid "n = n times, -1 = forever"
msgstr "n = n mal, -1 = für immer"
#: models.py:90
#, fuzzy
#| msgid "Stopped"
msgid "Stopped at"
msgstr "Gestoppt"
#: django_q/models.py:198
msgid "Next Run"
msgstr "Nächste Ausführung"
#: models.py:91
msgid "When the worker finished (success or failure)."
msgstr ""
#: django_q/models.py:205
msgid "Cron expression"
msgstr "Cron-Ausdruck"
#: django_q/models.py:235
msgid "Scheduled task"
msgstr "Geplante Aufgabe"
#: django_q/models.py:236
msgid "Scheduled tasks"
msgstr "Geplante Aufgaben"
#: django_q/models.py:262
msgid "Queued task"
msgstr "Eingereihte Aufgabe"
#: django_q/models.py:263
msgid "Queued tasks"
msgstr "Eingereihte Aufgaben"
#: django_q/monitor.py:62 django_q/monitor.py:339
msgid "Host"
msgstr "Host"
#: django_q/monitor.py:66 django_q/monitor.py:343 django_q/monitor.py:450
msgid "Id"
msgstr "Id"
#: django_q/monitor.py:70
msgid "State"
msgstr "Status"
#: django_q/monitor.py:74
msgid "Pool"
msgstr "Pool"
#: django_q/monitor.py:78
msgid "TQ"
msgstr "TQ"
#: django_q/monitor.py:82
msgid "RQ"
msgstr "RQ"
#: django_q/monitor.py:86
msgid "RC"
msgstr "RC"
#: django_q/monitor.py:90
msgid "Up"
msgstr "Up"
#: django_q/monitor.py:170 django_q/monitor.py:279
msgid "Queued"
msgstr "Eingereiht"
#: django_q/monitor.py:178
#: models.py:96 monitor_terminal.py:180
msgid "Success"
msgstr "Erfolg"
#: django_q/monitor.py:188 django_q/monitor.py:287
#: models.py:97
msgid "False if the task raised an exception or timed out."
msgstr ""
#: models.py:101
msgid "Attempt count"
msgstr ""
#: models.py:102
msgid "How many times execution was attempted."
msgstr ""
#: models.py:172
msgid "Task"
msgstr ""
#: models.py:173
msgid "Tasks"
msgstr ""
#: models.py:194
msgid "Successful task"
msgstr "Erfolgreiche Aufgabe"
#: models.py:195
msgid "Successful tasks"
msgstr "Erfolgreiche Aufgaben"
#: models.py:210
msgid "Failed task"
msgstr "Fehlgeschlagene Aufgabe"
#: models.py:211
msgid "Failed tasks"
msgstr "Fehlgeschlagene Aufgaben"
#: models.py:219 models.py:347
msgid "Please install croniter to enable cron expressions"
msgstr "Bitte installieren Sie croniter, um Cron-Ausdrücke zu aktivieren"
#: models.py:236
msgid "Optional label to identify this schedule in the admin."
msgstr ""
#: models.py:241
msgid "e.g. module.tasks.function"
msgstr ""
#: models.py:248
msgid "e.g. module.tasks.result_function"
msgstr ""
#: models.py:254
msgid "e.g. 1, 2, 'John'"
msgstr "zum Beispiel 1, 2, 'John'"
#: models.py:260
msgid "e.g. x=1, y=2, name='John'"
msgstr "zum Beispiel x=1, y=2, name='John'"
#: models.py:274
msgid "Once"
msgstr "Einmal"
#: models.py:275 models.py:296
msgid "Minutes"
msgstr "Minuten"
#: models.py:276
msgid "Hourly"
msgstr "Stündlich"
#: models.py:277
msgid "Daily"
msgstr "Täglich"
#: models.py:278
msgid "Weekly"
msgstr "Wöchentlich"
#: models.py:279
msgid "Biweekly"
msgstr "Zweiwöchentlich"
#: models.py:280
msgid "Monthly"
msgstr "Monatlich"
#: models.py:281
msgid "Bimonthly"
msgstr "Zweimonatlich"
#: models.py:282
msgid "Quarterly"
msgstr "Vierteljährlich"
#: models.py:283
msgid "Yearly"
msgstr "Jährlich"
#: models.py:284 models.py:313
msgid "Cron"
msgstr "Cron"
#: models.py:290
msgid "Schedule Type"
msgstr "Zeitplan-Typ"
#: models.py:291
msgid "How often this task should be enqueued."
msgstr ""
#: models.py:297
msgid "Number of minutes for the Minutes type"
msgstr "Anzahl Minuten für den Typ 'Minuten'"
#: models.py:300
msgid "Repeats"
msgstr "Wiederholungen"
#: models.py:300
msgid "n = n times, -1 = forever"
msgstr "n = n mal, -1 = für immer"
#: models.py:303
msgid "Next Run"
msgstr "Nächste Ausführung"
#: models.py:304
msgid "When this schedule runs next (stored in UTC)."
msgstr ""
#: models.py:314
msgid "Cron expression"
msgstr "Cron-Ausdruck"
#: models.py:320
msgid "Last task id"
msgstr ""
#: models.py:321
msgid "Id of the last task spawned from this schedule (read-only)."
msgstr ""
#: models.py:329 models.py:427
msgid "Name of the target cluster"
msgstr ""
#: models.py:336
msgid "Intended date kwarg"
msgstr ""
#: models.py:337
msgid "Name of kwarg to pass intended schedule date"
msgstr "Name des zu passierenden Kwargs vorgesehenes Datum"
#: models.py:418
msgid "Scheduled task"
msgstr "Geplante Aufgabe"
#: models.py:419
msgid "Scheduled tasks"
msgstr "Geplante Aufgaben"
#: models.py:426
#, fuzzy
#| msgid "Clusters"
msgid "Cluster key"
msgstr "Cluster"
#: models.py:430
msgid "Payload"
msgstr ""
#: models.py:431
msgid "Signed serialized task package (do not edit manually)."
msgstr ""
#: models.py:435
msgid "Lock until"
msgstr ""
#: models.py:436
msgid "Prevent any cluster from pulling until"
msgstr ""
#: models.py:474
#, fuzzy
#| msgid "Queued task"
msgid "Queue options"
msgstr "Eingereihte Aufgabe"
#: models.py:478
msgid "Queued task"
msgstr "Eingereihte Aufgabe"
#: models.py:479
msgid "Queued tasks"
msgstr "Eingereihte Aufgaben"
#: monitor.py:41
#, python-format
msgid "%(name)s monitoring at %(id)s"
msgstr "%(name)s beobachtet auf %(id)s"
#: monitor.py:61
#, python-format
msgid "Processed '%(info_name)s' (%(task_name)s)"
msgstr "[%(task_name)s] - '%(info_name)s' wurde verarbeitet"
#: monitor.py:67
#, python-format
msgid "Failed '%(info_name)s' (%(task_name)s) - %(task_result)s"
msgstr "'%(info_name)s' (%(task_name)s) ist fehlgeschlagen - %(task_result)s"
#: monitor.py:74
#, python-format
msgid "%(name)s stopped monitoring results"
msgstr "%(name)s überwacht keine Ergebnisse mehr"
#: monitor_terminal.py:64 monitor_terminal.py:348
msgid "Host"
msgstr "Host"
#: monitor_terminal.py:68 monitor_terminal.py:352 monitor_terminal.py:457
msgid "Id"
msgstr "Id"
#: monitor_terminal.py:72
msgid "State"
msgstr "Status"
#: monitor_terminal.py:76
msgid "Pool"
msgstr "Pool"
#: monitor_terminal.py:80
msgid "TQ"
msgstr "TQ"
#: monitor_terminal.py:84
msgid "RQ"
msgstr "RQ"
#: monitor_terminal.py:88
msgid "RC"
msgstr "RC"
#: monitor_terminal.py:92
msgid "Up"
msgstr "Up"
#: monitor_terminal.py:172 monitor_terminal.py:286
msgid "Queued"
msgstr "Eingereiht"
#: monitor_terminal.py:190 monitor_terminal.py:294
msgid "Failures"
msgstr "Fehlschläge"
#: django_q/monitor.py:199 django_q/monitor.py:485
#: monitor_terminal.py:201 monitor_terminal.py:496
msgid "[Press q to quit]"
msgstr "[Drücken Sie q zum Beenden]"
#: django_q/monitor.py:223
#: monitor_terminal.py:227
msgid "day"
msgstr "Tag"
#: django_q/monitor.py:244
#: monitor_terminal.py:248
msgid "second"
msgstr "Sekunde"
#: django_q/monitor.py:247
#: monitor_terminal.py:251
msgid "minute"
msgstr "Minute"
#: django_q/monitor.py:250
#: monitor_terminal.py:254
msgid "hour"
msgstr "Stunde"
#: django_q/monitor.py:260
#: monitor_terminal.py:263
#, python-format
msgid "-- %(prefix)s %(version)s on %(info)s --"
msgstr "-- %(prefix)s %(version)s auf %(info)s --"
#: django_q/monitor.py:266
#: monitor_terminal.py:273
msgid "Clusters"
msgstr "Cluster"
#: django_q/monitor.py:270
#: monitor_terminal.py:277
msgid "Workers"
msgstr "Arbeiter"
#: django_q/monitor.py:274
#: monitor_terminal.py:281
msgid "Restarts"
msgstr "Neustarts"
#: django_q/monitor.py:283
#: monitor_terminal.py:290
msgid "Successes"
msgstr "Erfolge"
#: django_q/monitor.py:292
#: monitor_terminal.py:299
msgid "Schedules"
msgstr "Zeitpläne"
#: django_q/monitor.py:296
#: monitor_terminal.py:303
#, python-format
msgid "Tasks/%(per)s"
msgstr "Aufgaben/%(per)s"
#: django_q/monitor.py:300
#: monitor_terminal.py:307
msgid "Avg time"
msgstr "Durchschnittl. Zeit"
#: django_q/monitor.py:348
#: monitor_terminal.py:357
msgid "Available (%)"
msgstr "Verfügbar (%)"
#: django_q/monitor.py:354
#: monitor_terminal.py:363
msgid "Available (MB)"
msgstr "Verfügbar (MB)"
#: django_q/monitor.py:359
#: monitor_terminal.py:368
msgid "Total (MB)"
msgstr "Insgesamt (MB)"
#: django_q/monitor.py:364
#: monitor_terminal.py:373
msgid "Sentinel (MB)"
msgstr "Sentinel (MB)"
#: django_q/monitor.py:370
#: monitor_terminal.py:379
msgid "Monitor (MB)"
msgstr "Monitor (MB)"
#: django_q/monitor.py:376
#: monitor_terminal.py:385
msgid "Workers (MB)"
msgstr "Arbeiter (MB)"
#: django_q/monitor.py:478
#: monitor_terminal.py:485
#, python-format
msgid "Available lowest (): %(memory_percent)s ((at)s)"
msgstr "Niedrigste verfügbar (): %(memory_percent)s ((at)s)"
#: django_q/monitor.py:496
#: monitor_terminal.py:507
msgid "No clusters appear to be running."
msgstr "Es scheinen keine Cluster zu laufen."
#: django_q/signals.py:22
#: pusher.py:40
#, python-format
#| msgid "%(process_name)s pushing tasks at %(id)s"
msgid "%(name)s pushing tasks at %(id)s"
msgstr "%(name)s veröffentlicht Aufagaben auf %(id)s"
#: pusher.py:67
#, python-format
msgid "queueing from %(list_key)s"
msgstr "Einreihen von %(list_key)s"
#: pusher.py:71
#, python-format
msgid "%(name)s stopped pushing tasks"
msgstr "%(name)s veröffentlicht keine Aufgaben mehr"
#: scheduler.py:110
#, python-format
msgid "%(process_name)s failed to create a task from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s konnte keine Aufgabe von Zeitplan [%(schedule)s] erstellen"
#: scheduler.py:121
#, fuzzy, python-format
#| msgid "%(process_name)s created a task from schedule [%(schedule)s]"
msgid ""
"%(process_name)s created task %(task_name)s from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s hat eine Aufgabe des Zeitplans [%(schedule)s] erstellt"
#: signals.py:22
#, python-format
msgid "malformed return hook '%(hook)s' for [%(name)s]"
msgstr "Ungültiger Return-Hook '%(hook)s' für [%(name)s]"
#: django_q/signals.py:30
#: signals.py:30
#, python-format
msgid "return hook %(hook)s failed on [%(name)s] because %(error)s"
msgstr "Return-Hook %(hook)s für [%(name)s] ist gescheitert: %(error)s"
#: timeout.py:30 timeout.py:43
msgid "SIGALARM is not available on your platform"
msgstr ""
#: worker.py:49
#, python-format
msgid "%(proc_name)s ready for work at %(id)s"
msgstr "%(proc_name)s ist bereit für Arbeit auf %(id)s"
#: worker.py:69
#, fuzzy, python-format
#| msgid "%(proc_name)s processing '%(func_name)s' (%(task_name)s)"
msgid "%(proc_name)s processing %(task_name)s '%(func_name)s'"
msgstr "%(proc_name)s verarbeitet '%(func_name)s' (%(task_name)s)"
#: worker.py:135
#, python-format
msgid "%(proc_name)s stopped doing work"
msgstr "%(proc_name)s hat die Arbeit beendet"
#, python-format
#~ msgid ""
#~ "Could not process '%(func_name)s'. Check the location of the function and "
#~ "the args/kwargs."
#~ msgstr ""
#~ "Konnte '%(func_name)s' nicht verarbeiten. Überprüfen Sie den Ort der "
#~ "Funktion und die args/kwargs."

View File

@@ -6,7 +6,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2022-11-12 01:47+0000\n"
"POT-Creation-Date: 2026-04-17 21:10+0000\n"
"PO-Revision-Date: 2018-08-05 18:28+0200\n"
"Last-Translator: Thierry BOULOGNE <contact@tng-concepts.com>\n"
"Language-Team: \n"
@@ -17,176 +17,118 @@ msgstr ""
"X-Generator: Poedit 2.1.1\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"
#: django_q/admin.py:43
#: admin.py:30
msgid "Resubmit selected tasks to queue"
msgstr "Resoumettre les tâches sélectionnées à la file d'attente"
#: django_q/admin.py:98 django_q/models.py:228
#: admin.py:64
msgid "Time Taken"
msgstr ""
#: admin.py:95
msgid "Short result"
msgstr ""
#: admin.py:142 models.py:412
#, fuzzy
#| msgid "Success"
msgid "success"
msgstr "succès"
#: django_q/admin.py:109 django_q/models.py:230
#: admin.py:154 models.py:414
msgid "last_run"
msgstr ""
#: django_q/cluster.py:77
#: cluster.py:75
#, python-format
msgid "Q Cluster %(name)s starting."
msgstr "Démarrage de Q Cluster-%(name)s."
#: django_q/cluster.py:85
#: cluster.py:83
#, python-format
msgid "Q Cluster %(name)s stopping."
msgstr "Arrêt de Q Cluster-%(name)s."
#: django_q/cluster.py:88
#: cluster.py:86
#, python-format
msgid "Q Cluster %(name)s has stopped."
msgstr "Q Cluster-%(name)s a été arrêté."
#: django_q/cluster.py:96
#: cluster.py:93
#, python-format
msgid "%(name)s got signal %(signal)s"
msgstr "%(name)s à reçu le signal %(signal)s"
#: django_q/cluster.py:219
#: cluster.py:224
#, python-format
msgid "reincarnated monitor %(name)s after sudden death"
msgstr "moniteur réintégré %(name)s après un arrêt intempestif"
msgstr "surveillant %(name)s réincarné après un arrêt intempestif"
#: django_q/cluster.py:222
#: cluster.py:230
#, python-format
msgid "reincarnated pusher %(name)s after sudden death"
msgstr "pousseur réintégré %(name)s après un arrêt intempestif"
msgstr "répartiteur %(name)s réincarné après un arrêt intempestif"
#: django_q/cluster.py:229
#: cluster.py:262
#, python-format
msgid ""
"reincarnated worker %(name)s after timeout while processing task "
"%(task_name)s"
msgstr ""
"processus %(name)s réincarné, délai de traitement dépassé pour la tâche "
"%(task_name)s"
#: cluster.py:265
#, python-format
msgid "reincarnated worker %(name)s after timeout"
msgstr "processus réintégré %(name)s après un arrêt une attente trop longue"
msgstr "processus %(name)s réincarné, délai de traitement dépassé"
#: django_q/cluster.py:231
#: cluster.py:270
#, python-format
msgid "recycled worker %(name)s"
msgstr "processus recyclé %(name)s"
#: django_q/cluster.py:233
#: cluster.py:273
#, python-format
msgid "reincarnated worker %(name)s after death"
msgstr "processus réintégré %(name)s après arrêt"
#: django_q/cluster.py:256
#: cluster.py:297
#, python-format
msgid "%(name)s guarding cluster %(cluster_name)s"
msgstr "%(name)s surveillance du cluster à %(cluster_name)s"
#: django_q/cluster.py:261
#: cluster.py:307
#, python-format
msgid "Q Cluster %(cluster_name)s running."
msgstr "Démarrage de Q Cluster-%(cluster_name)s."
#: django_q/cluster.py:295
#: cluster.py:346
#, python-format
msgid "%(name)s stopping cluster processes"
msgstr "%(name)s arrêt des processus de cluster"
msgstr "%(name)s arrêt des processus du cluster"
#: django_q/cluster.py:320
#: cluster.py:371
#, python-format
msgid "%(name)s waiting for the monitor."
msgstr "%(name)s en attente du moniteur."
msgstr "%(name)s en attente du surveillant."
#: django_q/cluster.py:342
#, python-format
msgid "%(process_name)s pushing tasks at %(id)s"
msgstr "%(process_name)s tâche envoyé à %(id)s"
#: django_q/cluster.py:363
#, python-format
msgid "queueing from %(list_key)s"
msgstr "mise en file d'attente de %(list_key)s"
#: django_q/cluster.py:366
#, python-format
msgid "%(name)s stopped pushing tasks"
msgstr "%(name)s a cessé de pousser les tâches"
#: django_q/cluster.py:378
#, python-format
msgid "%(name)s monitoring at %(id)s"
msgstr "%(name)s Surveillance de %(id)s"
#: django_q/cluster.py:395
#, python-format
msgid "Processed '%(info_name)s' (%(task_name)s)"
msgstr "traité '%(info_name)s' (%(task_name)s)"
#: django_q/cluster.py:398
#, python-format
msgid "Failed '%(info_name)s' (%(task_name)s) - %(task_result)s"
msgstr "Manqué '%(info_name)s' (%(task_name)s) - %(task_result)s"
#: django_q/cluster.py:399
#, python-format
msgid "%(name)s stopped monitoring results"
msgstr "%(name)s arrêt des résultats de surveillance"
#: django_q/cluster.py:413
#, python-format
msgid "%(proc_name)s ready for work at %(id)s"
msgstr "%(proc_name)s prêt pour le travail à %(id)s"
#: django_q/cluster.py:425
#, python-format
msgid "%(proc_name)s processing '%(func_name)s' (%(task_name)s)"
msgstr "%(proc_name)s en traitement '%(func_name)s' (%(task_name)s)"
#: django_q/cluster.py:440
#, python-format
msgid ""
"Could not process '%(func_name)s'. Check the location of the function and "
"the args/kwargs."
msgstr ""
"Impossible de traiter '%(func_name)s'. Vérifiez l'emplacement de la fonction "
"et les args/kwargs."
#: django_q/cluster.py:456
#, python-format
msgid "%(proc_name)s stopped doing work"
msgstr "%(proc_name)s arrêté de travailler"
#: django_q/cluster.py:649 django_q/models.py:144
msgid "Please install croniter to enable cron expressions"
msgstr "Veuillez installer croniter pour activer les expressions croniques."
#: django_q/cluster.py:672
#, python-format
msgid "%(process_name)s failed to create a task from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s Echec de la création d'une tâche à partir de Schedule "
"[%(schedule)s]"
#: django_q/cluster.py:678
#, python-format
msgid "%(process_name)s created a task from schedule [%(schedule)s]"
msgstr "%(process_name)s a créé une tâche à partir de Schedule [%(schedule)s]"
#: django_q/cluster.py:718
#: cluster.py:394
msgid "Skipping cpu affinity because psutil was not found."
msgstr "Sauter l'affinité du processeur parce que psutil n'a pas été trouvé."
msgstr "L'affinité cpu ne sera pas définie car psutil n'a pas été trouvé."
#: django_q/cluster.py:723
#: cluster.py:399
msgid "Faking cpu affinity because it is not supported on this platform"
msgstr ""
"Simulation de l'affinité du processeur parce qu'elle n'est pas supportée sur "
"cette plateforme."
"Simulation de l'affinité cpu parce qu'elle n'est pas supportée sur cette "
"plateforme."
#: django_q/cluster.py:744
#: cluster.py:421
#, python-format
msgid "%(pid)s will use cpu %(affinity)s"
msgstr "%(pid)s utilisera le CPU %(affinity)s"
#: django_q/conf.py:85
#: conf.py:122
#, python-format
msgid ""
"SAVE_LIMIT_PER (%(option)s) is not a valid option. Options are: 'group', "
@@ -196,299 +138,560 @@ msgstr ""
"'group', 'name', 'func' et None. La valeur par défaut est None."
#. Translators: Cluster status descriptions
#: django_q/conf.py:194
#: conf.py:239
msgid "Starting"
msgstr "Démarrage"
#: django_q/conf.py:195
#: conf.py:240
msgid "Working"
msgstr "Actif"
#: django_q/conf.py:196
#: conf.py:241
msgid "Idle"
msgstr "En attente"
#: django_q/conf.py:197
#: conf.py:242
msgid "Stopped"
msgstr "Arrêté"
#: django_q/conf.py:198
#: conf.py:243
msgid "Stopping"
msgstr "En cours darrêt"
#. Translators: help text for qcluster management command
#: django_q/management/commands/qcluster.py:9
#: management/commands/qcluster.py:11
msgid "Starts a Django Q Cluster."
msgstr "Démarre un cluster Django Q."
#. Translators: help text for qinfo management command
#: django_q/management/commands/qinfo.py:11
#: management/commands/qinfo.py:11
msgid "General information over all clusters."
msgstr "Informations générales sur tous les clusters."
#. Translators: help text for qmemory management command
#: django_q/management/commands/qmemory.py:9
#: management/commands/qmemory.py:9
#, fuzzy
#| msgid "Monitors Q Cluster activity"
msgid "Monitors Q Cluster memory usage"
msgstr "Surveille l'utilisation de la mémoire du cluster Q"
msgstr "Surveille l'utilisation mémoire du Q cluster"
#. Translators: help text for qmonitor management command
#: django_q/management/commands/qmonitor.py:9
#: management/commands/qmonitor.py:9
msgid "Monitors Q Cluster activity"
msgstr "Activité du cluster Moniteur Q"
msgstr "Surveille l'activité de Q cluster"
#: django_q/models.py:119
#: models.py:30 models.py:469
msgid "Task id"
msgstr ""
#: models.py:31
msgid "32-character task identifier."
msgstr ""
#: models.py:36 models.py:235 models.py:470
msgid "Name"
msgstr ""
#: models.py:37
msgid "Optional human-readable name for lookup and display."
msgstr ""
#: models.py:41 models.py:240 models.py:468
msgid "Function"
msgstr ""
#: models.py:42
msgid "Dotted import path to the callable, e.g. myapp.tasks.job."
msgstr ""
#: models.py:47 models.py:247
msgid "Hook"
msgstr ""
#: models.py:48
msgid "Optional dotted path to a callback that receives the task result."
msgstr ""
#: models.py:53 models.py:253 models.py:472
msgid "Arguments"
msgstr ""
#: models.py:54
msgid "Pickled positional arguments (tuple)."
msgstr ""
#: models.py:59 models.py:259 models.py:473
msgid "Keyword arguments"
msgstr ""
#: models.py:60
msgid "Pickled keyword arguments (dict)."
msgstr ""
#: models.py:65
msgid "Result"
msgstr ""
#: models.py:66
msgid "Pickled return value or error payload."
msgstr ""
#: models.py:72 models.py:471
msgid "Group"
msgstr ""
#: models.py:73
msgid "Optional group id to aggregate related tasks."
msgstr ""
#: models.py:80 models.py:328
#, fuzzy
#| msgid "Clusters"
msgid "Cluster"
msgstr "Grappes"
#: models.py:81
msgid "Name of the cluster that executed this task."
msgstr ""
#: models.py:85
msgid "Started at"
msgstr ""
#: models.py:86
msgid "When the worker started executing the task."
msgstr ""
#: models.py:90
#, fuzzy
#| msgid "Stopped"
msgid "Stopped at"
msgstr "Arrêté"
#: models.py:91
msgid "When the worker finished (success or failure)."
msgstr ""
#: models.py:96 monitor_terminal.py:180
msgid "Success"
msgstr "Succès"
#: models.py:97
msgid "False if the task raised an exception or timed out."
msgstr ""
#: models.py:101
msgid "Attempt count"
msgstr ""
#: models.py:102
msgid "How many times execution was attempted."
msgstr ""
#: models.py:172
msgid "Task"
msgstr ""
#: models.py:173
msgid "Tasks"
msgstr ""
#: models.py:194
msgid "Successful task"
msgstr "Tâche réussie"
#: django_q/models.py:120
#: models.py:195
msgid "Successful tasks"
msgstr "Tâches réussies"
#: django_q/models.py:135
#: models.py:210
msgid "Failed task"
msgstr "Tâche échoué"
#: django_q/models.py:136
#: models.py:211
msgid "Failed tasks"
msgstr "Tâches échouées"
#: django_q/models.py:160
#: models.py:219 models.py:347
msgid "Please install croniter to enable cron expressions"
msgstr "Veuillez installer croniter pour activer les expressions cron."
#: models.py:236
msgid "Optional label to identify this schedule in the admin."
msgstr ""
#: models.py:241
msgid "e.g. module.tasks.function"
msgstr ""
#: models.py:248
msgid "e.g. module.tasks.result_function"
msgstr ""
#: models.py:254
msgid "e.g. 1, 2, 'John'"
msgstr "ex. 1, 2, Jean"
#: django_q/models.py:162
#: models.py:260
msgid "e.g. x=1, y=2, name='John'"
msgstr "p. ex. x = 1, y = 2, Nom = Jean"
#: django_q/models.py:176
#: models.py:274
msgid "Once"
msgstr "Une fois"
#: django_q/models.py:177
#: models.py:275 models.py:296
msgid "Minutes"
msgstr "Minutes"
#: django_q/models.py:178
#: models.py:276
msgid "Hourly"
msgstr "Toutes les heures"
#: django_q/models.py:179
#: models.py:277
msgid "Daily"
msgstr "Quotidien"
#: django_q/models.py:180
#: models.py:278
msgid "Weekly"
msgstr "Hebdomadaire"
#: django_q/models.py:181
#: models.py:279
#, fuzzy
#| msgid "Weekly"
msgid "Biweekly"
msgstr "Bihebdomadaire"
#: django_q/models.py:182
#: models.py:280
msgid "Monthly"
msgstr "Mensuel"
#: django_q/models.py:183
#: models.py:281
#, fuzzy
#| msgid "Monthly"
msgid "Bimonthly"
msgstr "Bimestriel"
#: django_q/models.py:184
#: models.py:282
msgid "Quarterly"
msgstr "Tous les quart-dheure"
msgstr "Trimestriel"
#: django_q/models.py:185
#: models.py:283
msgid "Yearly"
msgstr "Annuel"
#: django_q/models.py:186
#: models.py:284 models.py:313
msgid "Cron"
msgstr "Cron"
#: django_q/models.py:189
#: models.py:290
msgid "Schedule Type"
msgstr "Type de plannification"
#: django_q/models.py:192
#: models.py:291
msgid "How often this task should be enqueued."
msgstr ""
#: models.py:297
msgid "Number of minutes for the Minutes type"
msgstr "Nombre de minutes pour le type de minutes"
#: django_q/models.py:195
#: models.py:300
msgid "Repeats"
msgstr "Répéter"
#: django_q/models.py:195
#: models.py:300
msgid "n = n times, -1 = forever"
msgstr "n = n fois,-1 = Toujours"
#: django_q/models.py:198
#: models.py:303
msgid "Next Run"
msgstr "Prochaine exécution"
#: django_q/models.py:205
msgid "Cron expression"
msgstr "Expression du Cron"
#: models.py:304
msgid "When this schedule runs next (stored in UTC)."
msgstr ""
#: django_q/models.py:235
#: models.py:314
msgid "Cron expression"
msgstr "Expression Cron"
#: models.py:320
msgid "Last task id"
msgstr ""
#: models.py:321
msgid "Id of the last task spawned from this schedule (read-only)."
msgstr ""
#: models.py:329 models.py:427
msgid "Name of the target cluster"
msgstr ""
#: models.py:336
msgid "Intended date kwarg"
msgstr ""
#: models.py:337
msgid "Name of kwarg to pass intended schedule date"
msgstr "Nom du kwarg à passer Date prévue de l'horaire"
#: models.py:418
msgid "Scheduled task"
msgstr "Tâche planifiée"
#: django_q/models.py:236
#: models.py:419
msgid "Scheduled tasks"
msgstr "Tâches planifiées"
#: django_q/models.py:262
#: models.py:426
#, fuzzy
#| msgid "Clusters"
msgid "Cluster key"
msgstr "Grappes"
#: models.py:430
msgid "Payload"
msgstr ""
#: models.py:431
msgid "Signed serialized task package (do not edit manually)."
msgstr ""
#: models.py:435
msgid "Lock until"
msgstr ""
#: models.py:436
msgid "Prevent any cluster from pulling until"
msgstr ""
#: models.py:474
#, fuzzy
#| msgid "Queued task"
msgid "Queue options"
msgstr "Tâche en file d'attente"
#: models.py:478
msgid "Queued task"
msgstr "Tâche en file d'attente"
#: django_q/models.py:263
#: models.py:479
msgid "Queued tasks"
msgstr "Tâches en file d'attente"
#: django_q/monitor.py:62 django_q/monitor.py:339
#: monitor.py:41
#, python-format
msgid "%(name)s monitoring at %(id)s"
msgstr "%(name)s surveille les résultats %(id)s"
#: monitor.py:61
#, python-format
msgid "Processed '%(info_name)s' (%(task_name)s)"
msgstr "traité '%(info_name)s' (%(task_name)s)"
#: monitor.py:67
#, python-format
msgid "Failed '%(info_name)s' (%(task_name)s) - %(task_result)s"
msgstr "Manqué '%(info_name)s' (%(task_name)s) - %(task_result)s"
#: monitor.py:74
#, python-format
msgid "%(name)s stopped monitoring results"
msgstr "%(name)s a cessé de de surveiller les résultats"
#: monitor_terminal.py:64 monitor_terminal.py:348
msgid "Host"
msgstr "Hôte"
#: django_q/monitor.py:66 django_q/monitor.py:343 django_q/monitor.py:450
#: monitor_terminal.py:68 monitor_terminal.py:352 monitor_terminal.py:457
msgid "Id"
msgstr "Id"
#: django_q/monitor.py:70
#: monitor_terminal.py:72
msgid "State"
msgstr "Statut"
#: django_q/monitor.py:74
#: monitor_terminal.py:76
msgid "Pool"
msgstr "Piscine"
#: django_q/monitor.py:78
#: monitor_terminal.py:80
msgid "TQ"
msgstr "TQ"
#: django_q/monitor.py:82
#: monitor_terminal.py:84
msgid "RQ"
msgstr "RQ"
#: django_q/monitor.py:86
#: monitor_terminal.py:88
msgid "RC"
msgstr "RC"
#: django_q/monitor.py:90
#: monitor_terminal.py:92
msgid "Up"
msgstr "Haut"
#: django_q/monitor.py:170 django_q/monitor.py:279
#: monitor_terminal.py:172 monitor_terminal.py:286
msgid "Queued"
msgstr "En file d'attente"
#: django_q/monitor.py:178
msgid "Success"
msgstr "Succès"
#: django_q/monitor.py:188 django_q/monitor.py:287
#: monitor_terminal.py:190 monitor_terminal.py:294
msgid "Failures"
msgstr "Défaillances"
#: django_q/monitor.py:199 django_q/monitor.py:485
#: monitor_terminal.py:201 monitor_terminal.py:496
msgid "[Press q to quit]"
msgstr "[appuyez sur q pour quitter]"
#: django_q/monitor.py:223
#: monitor_terminal.py:227
msgid "day"
msgstr "jour"
#: django_q/monitor.py:244
#: monitor_terminal.py:248
msgid "second"
msgstr "seconde"
#: django_q/monitor.py:247
#: monitor_terminal.py:251
msgid "minute"
msgstr "minute"
#: django_q/monitor.py:250
#: monitor_terminal.py:254
msgid "hour"
msgstr "heure"
#: django_q/monitor.py:260
#: monitor_terminal.py:263
#, python-format
msgid "-- %(prefix)s %(version)s on %(info)s --"
msgstr "--%(prefix)s %(version)s sur %(info)s --"
#: django_q/monitor.py:266
#: monitor_terminal.py:273
msgid "Clusters"
msgstr "Grappes"
#: django_q/monitor.py:270
#: monitor_terminal.py:277
msgid "Workers"
msgstr "Processus"
#: django_q/monitor.py:274
#: monitor_terminal.py:281
msgid "Restarts"
msgstr "Redémarrages"
#: django_q/monitor.py:283
#: monitor_terminal.py:290
msgid "Successes"
msgstr "Succès"
#: django_q/monitor.py:292
#: monitor_terminal.py:299
msgid "Schedules"
msgstr "Planifications"
#: django_q/monitor.py:296
#: monitor_terminal.py:303
#, python-format
msgid "Tasks/%(per)s"
msgstr "Tâches/%(per)s"
#: django_q/monitor.py:300
#: monitor_terminal.py:307
msgid "Avg time"
msgstr "Temps Moyen"
#: django_q/monitor.py:348
#: monitor_terminal.py:357
msgid "Available (%)"
msgstr ""
#: django_q/monitor.py:354
#: monitor_terminal.py:363
msgid "Available (MB)"
msgstr "Disponible sur (MB)"
#: django_q/monitor.py:359
#: monitor_terminal.py:368
msgid "Total (MB)"
msgstr "Total (MB)"
#: django_q/monitor.py:364
#: monitor_terminal.py:373
msgid "Sentinel (MB)"
msgstr "Sentinel (MB)"
#: django_q/monitor.py:370
#: monitor_terminal.py:379
msgid "Monitor (MB)"
msgstr "Monitor (MB)"
#: django_q/monitor.py:376
#: monitor_terminal.py:385
#, fuzzy
#| msgid "Workers"
msgid "Workers (MB)"
msgstr "Processus (MB)"
#: django_q/monitor.py:478
#: monitor_terminal.py:485
#, python-format
msgid "Available lowest (): %(memory_percent)s ((at)s)"
msgstr "Disponible le plus bas () : %(memory_percent)s ((at)s)"
#: django_q/monitor.py:496
#: monitor_terminal.py:507
msgid "No clusters appear to be running."
msgstr "Aucun cluster ne semble être en cours d'exécution."
#: django_q/signals.py:22
#: pusher.py:40
#, python-format
msgid "%(name)s pushing tasks at %(id)s"
msgstr "%(name)s répartit les tâches %(id)s"
#: pusher.py:67
#, python-format
msgid "queueing from %(list_key)s"
msgstr "mise en file d'attente de %(list_key)s"
#: pusher.py:71
#, python-format
msgid "%(name)s stopped pushing tasks"
msgstr "%(name)s a cessé de répartir les tâches"
#: scheduler.py:110
#, python-format
msgid "%(process_name)s failed to create a task from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s Echec de la création d'une tâche à partir de Schedule "
"[%(schedule)s]"
#: scheduler.py:121
#, python-format
msgid ""
"%(process_name)s created task %(task_name)s from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s a créé la tâche %(task_name)s à partir de Schedule "
"[%(schedule)s]"
#: signals.py:22
#, python-format
msgid "malformed return hook '%(hook)s' for [%(name)s]"
msgstr "hook de retour mal formé' %(hook)s 'pour [%(name)s]"
msgstr "hook de retour '%(hook)s' mal formé pour [%(name)s]"
#: django_q/signals.py:30
#: signals.py:30
#, python-format
msgid "return hook %(hook)s failed on [%(name)s] because %(error)s"
msgstr "le crochet de retour %(hook)s a échoué sur [%(name)s] parce que %(error)s"
msgstr "hook de retour %(hook)s a échoué sur [%(name)s] à cause de %(error)s"
#: timeout.py:30 timeout.py:43
msgid "SIGALARM is not available on your platform"
msgstr ""
#: worker.py:49
#, python-format
msgid "%(proc_name)s ready for work at %(id)s"
msgstr "%(proc_name)s prêt pour le travail à %(id)s"
#: worker.py:69
#, fuzzy, python-format
msgid "%(proc_name)s processing %(task_name)s '%(func_name)s'"
msgstr "%(proc_name)s exécute %(task_name)s '%(func_name)s'"
#: worker.py:135
#, python-format
msgid "%(proc_name)s stopped doing work"
msgstr "%(proc_name)s a cessé de travailler"
#, python-format
#~ msgid ""
#~ "Could not process '%(func_name)s'. Check the location of the function and "
#~ "the args/kwargs."
#~ msgstr ""
#~ "Impossible de traiter '%(func_name)s'. Vérifiez l'emplacement de la "
#~ "fonction et les args/kwargs."

View File

@@ -0,0 +1,676 @@
# Russian translation for django-q.
# Copyright (C) Django Q authors
# This file is distributed under the same license as the django-q package.
#
msgid ""
msgstr ""
"Project-Id-Version: django-q\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-04-17 21:07+0000\n"
"PO-Revision-Date: 2026-04-04 12:00+0000\n"
"Last-Translator: \n"
"Language-Team: Russian\n"
"Language: ru\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=4; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && "
"n%10<=4 && (n%100<12 || n%100>14) ? 1 : n%10==0 || (n%10>=5 && n%10<=9) || "
"(n%100>=11 && n%100<=14)? 2 : 3);\n"
#: django_q/admin.py:30
msgid "Resubmit selected tasks to queue"
msgstr "Повторно отправить выбранные задачи в очередь"
#: django_q/admin.py:64
msgid "Time Taken"
msgstr "Время выполнения"
#: django_q/admin.py:95
msgid "Short result"
msgstr "Краткий результат"
#: django_q/admin.py:142 django_q/models.py:412
msgid "success"
msgstr "успех"
#: django_q/admin.py:154 django_q/models.py:414
msgid "last_run"
msgstr "последний запуск"
#: django_q/cluster.py:75
#, python-format
msgid "Q Cluster %(name)s starting."
msgstr "Запуск кластера Q %(name)s."
#: django_q/cluster.py:83
#, python-format
msgid "Q Cluster %(name)s stopping."
msgstr "Остановка кластера Q %(name)s."
#: django_q/cluster.py:86
#, python-format
msgid "Q Cluster %(name)s has stopped."
msgstr "Кластер Q %(name)s остановлен."
#: django_q/cluster.py:93
#, python-format
msgid "%(name)s got signal %(signal)s"
msgstr "%(name)s получил сигнал %(signal)s"
#: django_q/cluster.py:224
#, python-format
msgid "reincarnated monitor %(name)s after sudden death"
msgstr "монитор %(name)s перезапущен после аварийного завершения"
#: django_q/cluster.py:230
#, python-format
msgid "reincarnated pusher %(name)s after sudden death"
msgstr "пушер %(name)s перезапущен после аварийного завершения"
#: django_q/cluster.py:262
#, python-format
msgid ""
"reincarnated worker %(name)s after timeout while processing task "
"%(task_name)s"
msgstr ""
"воркер %(name)s перезапущен по таймауту при выполнении задачи %(task_name)s"
#: django_q/cluster.py:265
#, python-format
msgid "reincarnated worker %(name)s after timeout"
msgstr "воркер %(name)s перезапущен по таймауту"
#: django_q/cluster.py:270
#, python-format
msgid "recycled worker %(name)s"
msgstr "воркер %(name)s перезапущен (recycle)"
#: django_q/cluster.py:273
#, python-format
msgid "reincarnated worker %(name)s after death"
msgstr "воркер %(name)s перезапущен после сбоя"
#: django_q/cluster.py:297
#, python-format
msgid "%(name)s guarding cluster %(cluster_name)s"
msgstr "%(name)s обслуживает кластер %(cluster_name)s"
#: django_q/cluster.py:307
#, python-format
msgid "Q Cluster %(cluster_name)s running."
msgstr "Кластер Q %(cluster_name)s работает."
#: django_q/cluster.py:346
#, python-format
msgid "%(name)s stopping cluster processes"
msgstr "%(name)s останавливает процессы кластера"
#: django_q/cluster.py:371
#, python-format
msgid "%(name)s waiting for the monitor."
msgstr "%(name)s ожидает монитор."
#: django_q/cluster.py:394
msgid "Skipping cpu affinity because psutil was not found."
msgstr "Привязка к CPU пропущена: не найден psutil."
#: django_q/cluster.py:399
msgid "Faking cpu affinity because it is not supported on this platform"
msgstr "Имитация привязки к CPU: на этой платформе не поддерживается"
#: django_q/cluster.py:421
#, python-format
msgid "%(pid)s will use cpu %(affinity)s"
msgstr "%(pid)s будет использовать CPU %(affinity)s"
#: django_q/conf.py:122
#, python-format
msgid ""
"SAVE_LIMIT_PER (%(option)s) is not a valid option. Options are: 'group', "
"'name', 'func' and None. Default is None."
msgstr ""
"SAVE_LIMIT_PER (%(option)s) — недопустимое значение. Допустимо: «group», "
"«name», «func» и None. По умолчанию None."
#. Translators: Cluster status descriptions
#: django_q/conf.py:239
msgid "Starting"
msgstr "Запуск"
#: django_q/conf.py:240
msgid "Working"
msgstr "Работает"
#: django_q/conf.py:241
msgid "Idle"
msgstr "Простой"
#: django_q/conf.py:242
msgid "Stopped"
msgstr "Остановлен"
#: django_q/conf.py:243
msgid "Stopping"
msgstr "Останавливается"
#. Translators: help text for qcluster management command
#: django_q/management/commands/qcluster.py:11
msgid "Starts a Django Q Cluster."
msgstr "Запускает кластер Django Q."
#. Translators: help text for qinfo management command
#: django_q/management/commands/qinfo.py:11
msgid "General information over all clusters."
msgstr "Общая информация по всем кластерам."
#. Translators: help text for qmemory management command
#: django_q/management/commands/qmemory.py:9
msgid "Monitors Q Cluster memory usage"
msgstr "Мониторинг использования памяти кластером Q"
#. Translators: help text for qmonitor management command
#: django_q/management/commands/qmonitor.py:9
msgid "Monitors Q Cluster activity"
msgstr "Мониторинг активности кластера Q"
#: django_q/models.py:30 django_q/models.py:469
msgid "Task id"
msgstr "Идентификатор задачи"
#: django_q/models.py:31
msgid "32-character task identifier."
msgstr "32-символьный идентификатор задачи."
#: django_q/models.py:36 django_q/models.py:235 django_q/models.py:470
msgid "Name"
msgstr "Имя"
#: django_q/models.py:37
msgid "Optional human-readable name for lookup and display."
msgstr "Необязательное имя для поиска и отображения."
#: django_q/models.py:41 django_q/models.py:240 django_q/models.py:468
msgid "Function"
msgstr "Функция"
#: django_q/models.py:42
msgid "Dotted import path to the callable, e.g. myapp.tasks.job."
msgstr "Путь импорта к вызываемому объекту, напр. myapp.tasks.job."
#: django_q/models.py:47 django_q/models.py:247
msgid "Hook"
msgstr "Обработчик"
#: django_q/models.py:48
msgid "Optional dotted path to a callback that receives the task result."
msgstr "Необязательный путь к функции, получающей результат задачи."
#: django_q/models.py:53 django_q/models.py:253 django_q/models.py:472
msgid "Arguments"
msgstr "Аргументы"
#: django_q/models.py:54
msgid "Pickled positional arguments (tuple)."
msgstr "Сериализованные позиционные аргументы (tuple)."
#: django_q/models.py:59 django_q/models.py:259 django_q/models.py:473
msgid "Keyword arguments"
msgstr "Именованные аргументы"
#: django_q/models.py:60
msgid "Pickled keyword arguments (dict)."
msgstr "Сериализованные именованные аргументы (dict)."
#: django_q/models.py:65
msgid "Result"
msgstr "Результат"
#: django_q/models.py:66
msgid "Pickled return value or error payload."
msgstr "Сериализованный результат или данные об ошибке."
#: django_q/models.py:72 django_q/models.py:471
msgid "Group"
msgstr "Группа"
#: django_q/models.py:73
msgid "Optional group id to aggregate related tasks."
msgstr "Необязательный идентификатор группы связанных задач."
#: django_q/models.py:80 django_q/models.py:328
msgid "Cluster"
msgstr "Кластер"
#: django_q/models.py:81
msgid "Name of the cluster that executed this task."
msgstr "Имя кластера, выполнившего эту задачу."
#: django_q/models.py:85
msgid "Started at"
msgstr "Начало выполнения"
#: django_q/models.py:86
msgid "When the worker started executing the task."
msgstr "Когда воркер начал выполнение задачи."
#: django_q/models.py:90
msgid "Stopped at"
msgstr "Окончание выполнения"
#: django_q/models.py:91
msgid "When the worker finished (success or failure)."
msgstr "Когда воркер завершил выполнение (успех или ошибка)."
#: django_q/models.py:96 django_q/monitor_terminal.py:180
msgid "Success"
msgstr "Успех"
#: django_q/models.py:97
msgid "False if the task raised an exception or timed out."
msgstr "Ложь, если задача завершилась исключением или таймаутом."
#: django_q/models.py:101
msgid "Attempt count"
msgstr "Число попыток"
#: django_q/models.py:102
msgid "How many times execution was attempted."
msgstr "Сколько раз выполнялась попытка запуска."
#: django_q/models.py:172
msgid "Task"
msgstr "Задача"
#: django_q/models.py:173
msgid "Tasks"
msgstr "Задачи"
#: django_q/models.py:194
msgid "Successful task"
msgstr "Успешная задача"
#: django_q/models.py:195
msgid "Successful tasks"
msgstr "Успешные задачи"
#: django_q/models.py:210
msgid "Failed task"
msgstr "Неудачная задача"
#: django_q/models.py:211
msgid "Failed tasks"
msgstr "Неудачные задачи"
#: django_q/models.py:219 django_q/models.py:347
msgid "Please install croniter to enable cron expressions"
msgstr "Установите croniter для поддержки cron-выражений"
#: django_q/models.py:236
msgid "Optional label to identify this schedule in the admin."
msgstr "Необязательная подпись для расписания в админке."
#: django_q/models.py:241
msgid "e.g. module.tasks.function"
msgstr "например: module.tasks.function"
#: django_q/models.py:248
msgid "e.g. module.tasks.result_function"
msgstr "например: module.tasks.result_function "
#: django_q/models.py:254
msgid "e.g. 1, 2, 'John'"
msgstr "например: 1, 2, 'Алимхан'"
#: django_q/models.py:260
msgid "e.g. x=1, y=2, name='John'"
msgstr "например: x=1, y=2, name='Алимхан'"
#: django_q/models.py:274
msgid "Once"
msgstr "Один раз"
#: django_q/models.py:275 django_q/models.py:296
msgid "Minutes"
msgstr "Минуты"
#: django_q/models.py:276
msgid "Hourly"
msgstr "Ежечасно"
#: django_q/models.py:277
msgid "Daily"
msgstr "Ежедневно"
#: django_q/models.py:278
msgid "Weekly"
msgstr "Еженедельно"
#: django_q/models.py:279
msgid "Biweekly"
msgstr "Раз в две недели"
#: django_q/models.py:280
msgid "Monthly"
msgstr "Ежемесячно"
#: django_q/models.py:281
msgid "Bimonthly"
msgstr "Раз в два месяца"
#: django_q/models.py:282
msgid "Quarterly"
msgstr "Ежеквартально"
#: django_q/models.py:283
msgid "Yearly"
msgstr "Ежегодно"
#: django_q/models.py:284 django_q/models.py:313
msgid "Cron"
msgstr "Cron"
#: django_q/models.py:290
msgid "Schedule Type"
msgstr "Тип расписания"
#: django_q/models.py:291
msgid "How often this task should be enqueued."
msgstr "Как часто ставить эту задачу в очередь."
#: django_q/models.py:297
msgid "Number of minutes for the Minutes type"
msgstr "Число минут для типа «Минуты»"
#: django_q/models.py:300
msgid "Repeats"
msgstr "Повторы"
#: django_q/models.py:300
msgid "n = n times, -1 = forever"
msgstr "n — число повторов, -1 — бесконечно"
#: django_q/models.py:303
msgid "Next Run"
msgstr "Следующий запуск"
#: django_q/models.py:304
msgid "When this schedule runs next (stored in UTC)."
msgstr "Когда следующий запуск по расписанию (в UTC)."
#: django_q/models.py:314
msgid "Cron expression"
msgstr "Cron-выражение"
#: django_q/models.py:320
msgid "Last task id"
msgstr "ID последней задачи"
#: django_q/models.py:321
msgid "Id of the last task spawned from this schedule (read-only)."
msgstr "Идентификатор последней задачи по этому расписанию (read-only)."
#: django_q/models.py:329 django_q/models.py:427
msgid "Name of the target cluster"
msgstr "Имя целевого кластера"
#: django_q/models.py:336
msgid "Intended date kwarg"
msgstr "Имя аргумента для даты запуска"
#: django_q/models.py:337
msgid "Name of kwarg to pass intended schedule date"
msgstr "Имя kwargs для передачи запланированной даты"
#: django_q/models.py:418
msgid "Scheduled task"
msgstr "Запланированная задача"
#: django_q/models.py:419
msgid "Scheduled tasks"
msgstr "Запланированные задачи"
#: django_q/models.py:426
msgid "Cluster key"
msgstr "Ключ кластера"
#: django_q/models.py:430
msgid "Payload"
msgstr "Данные задачи"
#: django_q/models.py:431
msgid "Signed serialized task package (do not edit manually)."
msgstr "Подписанный сериализованный пакет задачи (не редактировать вручную)."
#: django_q/models.py:435
msgid "Lock until"
msgstr "Блокировка до"
#: django_q/models.py:436
msgid "Prevent any cluster from pulling until"
msgstr "Не забирать задачи кластерами до"
#: django_q/models.py:474
msgid "Queue options"
msgstr "Дополнительные параметры очереди"
#: django_q/models.py:478
msgid "Queued task"
msgstr "Задача в очереди"
#: django_q/models.py:479
msgid "Queued tasks"
msgstr "Задачи в очереди"
#: django_q/monitor.py:41
#, python-format
msgid "%(name)s monitoring at %(id)s"
msgstr "%(name)s мониторинг на %(id)s"
#: django_q/monitor.py:61
#, python-format
msgid "Processed '%(info_name)s' (%(task_name)s)"
msgstr "Обработано «%(info_name)s» (%(task_name)s)"
#: django_q/monitor.py:67
#, python-format
msgid "Failed '%(info_name)s' (%(task_name)s) - %(task_result)s"
msgstr "Ошибка «%(info_name)s» (%(task_name)s) — %(task_result)s"
#: django_q/monitor.py:74
#, python-format
msgid "%(name)s stopped monitoring results"
msgstr "%(name)s прекратил отслеживание результатов"
#: django_q/monitor_terminal.py:64 django_q/monitor_terminal.py:348
msgid "Host"
msgstr "Хост"
#: django_q/monitor_terminal.py:68 django_q/monitor_terminal.py:352
#: django_q/monitor_terminal.py:457
msgid "Id"
msgstr "Id"
#: django_q/monitor_terminal.py:72
msgid "State"
msgstr "Состояние"
#: django_q/monitor_terminal.py:76
msgid "Pool"
msgstr "Пул"
#: django_q/monitor_terminal.py:80
msgid "TQ"
msgstr "TQ"
#: django_q/monitor_terminal.py:84
msgid "RQ"
msgstr "RQ"
#: django_q/monitor_terminal.py:88
msgid "RC"
msgstr "RC"
#: django_q/monitor_terminal.py:92
msgid "Up"
msgstr "Аптайм"
#: django_q/monitor_terminal.py:172 django_q/monitor_terminal.py:286
msgid "Queued"
msgstr "В очереди"
#: django_q/monitor_terminal.py:190 django_q/monitor_terminal.py:294
msgid "Failures"
msgstr "Сбои"
#: django_q/monitor_terminal.py:201 django_q/monitor_terminal.py:496
msgid "[Press q to quit]"
msgstr "[Нажмите q для выхода]"
#: django_q/monitor_terminal.py:227
msgid "day"
msgstr "день"
#: django_q/monitor_terminal.py:248
msgid "second"
msgstr "секунда"
#: django_q/monitor_terminal.py:251
msgid "minute"
msgstr "минута"
#: django_q/monitor_terminal.py:254
msgid "hour"
msgstr "час"
#: django_q/monitor_terminal.py:263
#, python-format
msgid "-- %(prefix)s %(version)s on %(info)s --"
msgstr "-- %(prefix)s %(version)s на %(info)s --"
#: django_q/monitor_terminal.py:273
msgid "Clusters"
msgstr "Кластеры"
#: django_q/monitor_terminal.py:277
msgid "Workers"
msgstr "Воркеры"
#: django_q/monitor_terminal.py:281
msgid "Restarts"
msgstr "Перезапуски"
#: django_q/monitor_terminal.py:290
msgid "Successes"
msgstr "Успехи"
#: django_q/monitor_terminal.py:299
msgid "Schedules"
msgstr "Расписания"
#: django_q/monitor_terminal.py:303
#, python-format
msgid "Tasks/%(per)s"
msgstr "Задач/%(per)s"
#: django_q/monitor_terminal.py:307
msgid "Avg time"
msgstr "Среднее время"
#: django_q/monitor_terminal.py:357
msgid "Available (%)"
msgstr "Доступно (%)"
#: django_q/monitor_terminal.py:363
msgid "Available (MB)"
msgstr "Доступно (МБ)"
#: django_q/monitor_terminal.py:368
msgid "Total (MB)"
msgstr "Всего (МБ)"
#: django_q/monitor_terminal.py:373
msgid "Sentinel (MB)"
msgstr "Sentinel (МБ)"
#: django_q/monitor_terminal.py:379
msgid "Monitor (MB)"
msgstr "Монитор (МБ)"
#: django_q/monitor_terminal.py:385
msgid "Workers (MB)"
msgstr "Воркеры (МБ)"
#: django_q/monitor_terminal.py:485
#, python-format
msgid "Available lowest (): %(memory_percent)s ((at)s)"
msgstr "Минимум доступно (): %(memory_percent)s ((at)s)"
#: django_q/monitor_terminal.py:507
msgid "No clusters appear to be running."
msgstr "Похоже, ни один кластер не запущен."
#: django_q/pusher.py:40
#, python-format
msgid "%(name)s pushing tasks at %(id)s"
msgstr "%(name)s отправляет задачи на %(id)s"
#: django_q/pusher.py:67
#, python-format
msgid "queueing from %(list_key)s"
msgstr "очередь из %(list_key)s"
#: django_q/pusher.py:71
#, python-format
msgid "%(name)s stopped pushing tasks"
msgstr "%(name)s прекратил отправку задач"
#: django_q/scheduler.py:110
#, python-format
msgid "%(process_name)s failed to create a task from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s не удалось создать задачу по расписанию [%(schedule)s]"
#: django_q/scheduler.py:121
#, python-format
msgid ""
"%(process_name)s created task %(task_name)s from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s создал задачу %(task_name)s по расписанию [%(schedule)s]"
#: django_q/signals.py:22
#, python-format
msgid "malformed return hook '%(hook)s' for [%(name)s]"
msgstr "некорректный return hook «%(hook)s» для [%(name)s]"
#: django_q/signals.py:30
#, python-format
msgid "return hook %(hook)s failed on [%(name)s] because %(error)s"
msgstr "return hook %(hook)s для [%(name)s] завершился ошибкой: %(error)s"
#: django_q/timeout.py:30 django_q/timeout.py:43
msgid "SIGALARM is not available on your platform"
msgstr "SIGALRM недоступен на вашей платформе"
#: django_q/worker.py:49
#, python-format
msgid "%(proc_name)s ready for work at %(id)s"
msgstr "%(proc_name)s готов к работе на %(id)s"
#: django_q/worker.py:69
#, python-format
msgid "%(proc_name)s processing %(task_name)s '%(func_name)s'"
msgstr "%(proc_name)s выполняет %(task_name)s «%(func_name)s»"
#: django_q/worker.py:135
#, python-format
msgid "%(proc_name)s stopped doing work"
msgstr "%(proc_name)s прекратил работу"
#, python-format
#~ msgid ""
#~ "Could not process '%(func_name)s'. Check the location of the function and "
#~ "the args/kwargs."
#~ msgstr ""
#~ "Не удалось обработать «%(func_name)s». Проверьте путь к функции и args/"
#~ "kwargs."

View File

@@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2022-11-12 01:47+0000\n"
"POT-Creation-Date: 2026-04-17 21:10+0000\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: Ethem Güner <ethemguener@gmail.com>\n"
"Language-Team: \n"
@@ -18,172 +18,115 @@ msgstr ""
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"
#: django_q/admin.py:43
#: admin.py:30
msgid "Resubmit selected tasks to queue"
msgstr "Seçili işleri kuyruğa tekrar gönder"
#: django_q/admin.py:98 django_q/models.py:228
#: admin.py:64
msgid "Time Taken"
msgstr ""
#: admin.py:95
msgid "Short result"
msgstr ""
#: admin.py:142 models.py:412
#, fuzzy
#| msgid "Success"
msgid "success"
msgstr "başarılı olanlar"
#: django_q/admin.py:109 django_q/models.py:230
#: admin.py:154 models.py:414
msgid "last_run"
msgstr ""
#: django_q/cluster.py:77
#: cluster.py:75
#, python-format
msgid "Q Cluster %(name)s starting."
msgstr "Q Cluster %(name)s başlatılıyor."
#: django_q/cluster.py:85
#: cluster.py:83
#, python-format
msgid "Q Cluster %(name)s stopping."
msgstr "Q Cluster %(name)s durduruluyor."
#: django_q/cluster.py:88
#: cluster.py:86
#, python-format
msgid "Q Cluster %(name)s has stopped."
msgstr "Q Cluster %(name)s durduruldu."
#: django_q/cluster.py:96
#: cluster.py:93
#, python-format
msgid "%(name)s got signal %(signal)s"
msgstr "%(name)s, %(signal)s pid'inde izleniyor/monitoring yapılıyor."
#: django_q/cluster.py:219
#: cluster.py:224
#, python-format
msgid "reincarnated monitor %(name)s after sudden death"
msgstr "Monitor %(name)s ani ölüm sonrası tekrar dirildi"
#: django_q/cluster.py:222
#: cluster.py:230
#, python-format
msgid "reincarnated pusher %(name)s after sudden death"
msgstr "Pusher %(name)s ani ölüm sonrası tekrar dirildi"
#: django_q/cluster.py:229
#: cluster.py:262
#, fuzzy, python-format
#| msgid "reincarnated worker %(name)s after timeout"
msgid ""
"reincarnated worker %(name)s after timeout while processing task "
"%(task_name)s"
msgstr "Worker %(name)s zaman aşımı sonrası tekrar dirildi"
#: cluster.py:265
#, python-format
msgid "reincarnated worker %(name)s after timeout"
msgstr "Worker %(name)s zaman aşımı sonrası tekrar dirildi"
#: django_q/cluster.py:231
#: cluster.py:270
#, python-format
msgid "recycled worker %(name)s"
msgstr "Worker %(name)s geri döndürüldü"
#: django_q/cluster.py:233
#: cluster.py:273
#, python-format
msgid "reincarnated worker %(name)s after death"
msgstr "Worker %(name)s ani ölüm sonrası tekrar dirildi"
#: django_q/cluster.py:256
#: cluster.py:297
#, python-format
msgid "%(name)s guarding cluster %(cluster_name)s"
msgstr "%(name)s, %(cluster_name)s cluster'ını koruyor"
#: django_q/cluster.py:261
#: cluster.py:307
#, python-format
msgid "Q Cluster %(cluster_name)s running."
msgstr "Q Cluster %(cluster_name)s başlatılıyor."
#: django_q/cluster.py:295
#: cluster.py:346
#, python-format
msgid "%(name)s stopping cluster processes"
msgstr "Cluster %(name)s işlemleri durduruluyor."
#: django_q/cluster.py:320
#: cluster.py:371
#, python-format
msgid "%(name)s waiting for the monitor."
msgstr "%(name)s monitor için bekliyor."
#: django_q/cluster.py:342
#, python-format
msgid "%(process_name)s pushing tasks at %(id)s"
msgstr "%(process_name)s, işleri %(id)s pid'ine gönderiyor."
#: django_q/cluster.py:363
#, python-format
msgid "queueing from %(list_key)s"
msgstr ""
#: django_q/cluster.py:366
#, python-format
msgid "%(name)s stopped pushing tasks"
msgstr "%(name)s işleri göndermeyi durdurdu"
#: django_q/cluster.py:378
#, python-format
msgid "%(name)s monitoring at %(id)s"
msgstr "%(name)s, %(id)s pid'inde izleniyor/monitoring yapılıyor."
#: django_q/cluster.py:395
#, python-format
msgid "Processed '%(info_name)s' (%(task_name)s)"
msgstr "[%(task_name)s] - '%(info_name)s işlendi."
#: django_q/cluster.py:398
#, python-format
msgid "Failed '%(info_name)s' (%(task_name)s) - %(task_result)s"
msgstr "[%(task_name)s] - '%(info_name)s' - %(task_result)s başarısız oldu"
#: django_q/cluster.py:399
#, python-format
msgid "%(name)s stopped monitoring results"
msgstr "%(name)s sonuçları göstermeyi bıraktı"
#: django_q/cluster.py:413
#, python-format
msgid "%(proc_name)s ready for work at %(id)s"
msgstr "%(proc_name)s, %(id)s pid'inde çalışmaya hazır"
#: django_q/cluster.py:425
#, python-format
msgid "%(proc_name)s processing '%(func_name)s' (%(task_name)s)"
msgstr "%(proc_name)s, '%(func_name)s' [%(task_name)s] işlerini işiyor"
#: django_q/cluster.py:440
#, python-format
msgid ""
"Could not process '%(func_name)s'. Check the location of the function and "
"the args/kwargs."
msgstr ""
"%(func_name)s' işlenemedi. İşlevin konumunu ve args/kwargs öğelerini kontrol "
"edin."
#: django_q/cluster.py:456
#, python-format
msgid "%(proc_name)s stopped doing work"
msgstr "%(proc_name)s çalışmayı bıraktı"
#: django_q/cluster.py:649 django_q/models.py:144
msgid "Please install croniter to enable cron expressions"
msgstr "Cron expressions'ları açmak için croniter yükleyin"
#: django_q/cluster.py:672
#, python-format
msgid "%(process_name)s failed to create a task from schedule [%(schedule)s]"
msgstr "%(process_name)s programdan bir görev oluşturamadı [%(schedule)s]"
#: django_q/cluster.py:678
#, python-format
msgid "%(process_name)s created a task from schedule [%(schedule)s]"
msgstr "%(process_name)s programdan bir görev oluşturamadı [%(schedule)s]"
#: django_q/cluster.py:718
#: cluster.py:394
msgid "Skipping cpu affinity because psutil was not found."
msgstr "Psutil bulunamadığı için cpu benzeşimi atlanıyor."
#: django_q/cluster.py:723
#: cluster.py:399
msgid "Faking cpu affinity because it is not supported on this platform"
msgstr "Bu platformda desteklenmediği için sahte cpu benzeşimi"
#: django_q/cluster.py:744
#: cluster.py:421
#, python-format
msgid "%(pid)s will use cpu %(affinity)s"
msgstr "%(pid)s cpu %(affinity)s kullanacaktır"
#: django_q/conf.py:85
#: conf.py:122
#, python-format
msgid ""
"SAVE_LIMIT_PER (%(option)s) is not a valid option. Options are: 'group', "
@@ -193,295 +136,549 @@ msgstr ""
"'group', 'name', 'func' ve None. Varsayılan değer None'dır."
#. Translators: Cluster status descriptions
#: django_q/conf.py:194
#: conf.py:239
msgid "Starting"
msgstr "Başlıyor"
#: django_q/conf.py:195
#: conf.py:240
msgid "Working"
msgstr "Çalışıyor"
#: django_q/conf.py:196
#: conf.py:241
msgid "Idle"
msgstr "Boşta"
#: django_q/conf.py:197
#: conf.py:242
msgid "Stopped"
msgstr "Durdu"
#: django_q/conf.py:198
#: conf.py:243
msgid "Stopping"
msgstr "Durduruluyor"
#. Translators: help text for qcluster management command
#: django_q/management/commands/qcluster.py:9
#: management/commands/qcluster.py:11
msgid "Starts a Django Q Cluster."
msgstr "Bir Django Q Cluster çalıştırır."
#. Translators: help text for qinfo management command
#: django_q/management/commands/qinfo.py:11
#: management/commands/qinfo.py:11
msgid "General information over all clusters."
msgstr "Tüm cluster'lar için genel bilgiler."
#. Translators: help text for qmemory management command
#: django_q/management/commands/qmemory.py:9
#: management/commands/qmemory.py:9
msgid "Monitors Q Cluster memory usage"
msgstr "Q Cluster'ın bellek kullanımını izler"
#. Translators: help text for qmonitor management command
#: django_q/management/commands/qmonitor.py:9
#: management/commands/qmonitor.py:9
msgid "Monitors Q Cluster activity"
msgstr "Q Cluster'ın aktivitelerini izler"
#: django_q/models.py:119
#: models.py:30 models.py:469
msgid "Task id"
msgstr ""
#: models.py:31
msgid "32-character task identifier."
msgstr ""
#: models.py:36 models.py:235 models.py:470
msgid "Name"
msgstr ""
#: models.py:37
msgid "Optional human-readable name for lookup and display."
msgstr ""
#: models.py:41 models.py:240 models.py:468
msgid "Function"
msgstr ""
#: models.py:42
msgid "Dotted import path to the callable, e.g. myapp.tasks.job."
msgstr ""
#: models.py:47 models.py:247
msgid "Hook"
msgstr ""
#: models.py:48
msgid "Optional dotted path to a callback that receives the task result."
msgstr ""
#: models.py:53 models.py:253 models.py:472
msgid "Arguments"
msgstr ""
#: models.py:54
msgid "Pickled positional arguments (tuple)."
msgstr ""
#: models.py:59 models.py:259 models.py:473
msgid "Keyword arguments"
msgstr ""
#: models.py:60
msgid "Pickled keyword arguments (dict)."
msgstr ""
#: models.py:65
msgid "Result"
msgstr ""
#: models.py:66
msgid "Pickled return value or error payload."
msgstr ""
#: models.py:72 models.py:471
msgid "Group"
msgstr ""
#: models.py:73
msgid "Optional group id to aggregate related tasks."
msgstr ""
#: models.py:80 models.py:328
msgid "Cluster"
msgstr ""
#: models.py:81
msgid "Name of the cluster that executed this task."
msgstr ""
#: models.py:85
msgid "Started at"
msgstr ""
#: models.py:86
msgid "When the worker started executing the task."
msgstr ""
#: models.py:90
#, fuzzy
#| msgid "Stopped"
msgid "Stopped at"
msgstr "Durdu"
#: models.py:91
msgid "When the worker finished (success or failure)."
msgstr ""
#: models.py:96 monitor_terminal.py:180
msgid "Success"
msgstr "Başarılı olanlar"
#: models.py:97
msgid "False if the task raised an exception or timed out."
msgstr ""
#: models.py:101
msgid "Attempt count"
msgstr ""
#: models.py:102
msgid "How many times execution was attempted."
msgstr ""
#: models.py:172
msgid "Task"
msgstr ""
#: models.py:173
msgid "Tasks"
msgstr ""
#: models.py:194
msgid "Successful task"
msgstr "Başarılı iş"
#: django_q/models.py:120
#: models.py:195
msgid "Successful tasks"
msgstr "Başarılı işler"
#: django_q/models.py:135
#: models.py:210
msgid "Failed task"
msgstr "Başarısız iş"
#: django_q/models.py:136
#: models.py:211
msgid "Failed tasks"
msgstr "Başarısız işler"
#: django_q/models.py:160
#: models.py:219 models.py:347
msgid "Please install croniter to enable cron expressions"
msgstr "Cron expressions'ları açmak için croniter yükleyin"
#: models.py:236
msgid "Optional label to identify this schedule in the admin."
msgstr ""
#: models.py:241
msgid "e.g. module.tasks.function"
msgstr ""
#: models.py:248
msgid "e.g. module.tasks.result_function"
msgstr ""
#: models.py:254
msgid "e.g. 1, 2, 'John'"
msgstr "Örneğin: 1, 2, 'Melih'"
#: django_q/models.py:162
#: models.py:260
msgid "e.g. x=1, y=2, name='John'"
msgstr "Örneğin: x=1, y=2, name='Melih'"
#: django_q/models.py:176
#: models.py:274
msgid "Once"
msgstr "Bir kere"
#: django_q/models.py:177
#: models.py:275 models.py:296
msgid "Minutes"
msgstr "Dakika"
#: django_q/models.py:178
#: models.py:276
msgid "Hourly"
msgstr "Saatlik"
#: django_q/models.py:179
#: models.py:277
msgid "Daily"
msgstr "Günlük"
#: django_q/models.py:180
#: models.py:278
msgid "Weekly"
msgstr "Haftalık"
#: django_q/models.py:181
#: models.py:279
#, fuzzy
#| msgid "Weekly"
msgid "Biweekly"
msgstr "İki haftada bir"
#: django_q/models.py:182
#: models.py:280
msgid "Monthly"
msgstr "Aylık"
#: django_q/models.py:183
#: models.py:281
#, fuzzy
#| msgid "Monthly"
msgid "Bimonthly"
msgstr "İki ayda bir"
#: django_q/models.py:184
#: models.py:282
msgid "Quarterly"
msgstr "Bir Çeyrek (3 Ay)"
#: django_q/models.py:185
#: models.py:283
msgid "Yearly"
msgstr "Yıllık"
#: django_q/models.py:186
#: models.py:284 models.py:313
msgid "Cron"
msgstr ""
#: django_q/models.py:189
#: models.py:290
msgid "Schedule Type"
msgstr "Zamanlama Tipi"
#: django_q/models.py:192
#: models.py:291
msgid "How often this task should be enqueued."
msgstr ""
#: models.py:297
msgid "Number of minutes for the Minutes type"
msgstr "Dakika tipine göre dakika sayısı"
#: django_q/models.py:195
#: models.py:300
msgid "Repeats"
msgstr "Tekrar eder"
#: django_q/models.py:195
#: models.py:300
msgid "n = n times, -1 = forever"
msgstr "n = n kere, -1 = sonsuza kadar"
#: django_q/models.py:198
#: models.py:303
msgid "Next Run"
msgstr "Bir dahaki çalışma tarihi"
#: django_q/models.py:205
#: models.py:304
msgid "When this schedule runs next (stored in UTC)."
msgstr ""
#: models.py:314
msgid "Cron expression"
msgstr ""
#: django_q/models.py:235
#: models.py:320
msgid "Last task id"
msgstr ""
#: models.py:321
msgid "Id of the last task spawned from this schedule (read-only)."
msgstr ""
#: models.py:329 models.py:427
msgid "Name of the target cluster"
msgstr ""
#: models.py:336
msgid "Intended date kwarg"
msgstr ""
#: models.py:337
msgid "Name of kwarg to pass intended schedule date"
msgstr "Geçilecek kwarg'ın adı öngörülen program tarihi"
#: models.py:418
msgid "Scheduled task"
msgstr "Zamanlanmış iş"
#: django_q/models.py:236
#: models.py:419
msgid "Scheduled tasks"
msgstr "Zamanlanmış işler"
#: django_q/models.py:262
#: models.py:426
msgid "Cluster key"
msgstr ""
#: models.py:430
msgid "Payload"
msgstr ""
#: models.py:431
msgid "Signed serialized task package (do not edit manually)."
msgstr ""
#: models.py:435
msgid "Lock until"
msgstr ""
#: models.py:436
msgid "Prevent any cluster from pulling until"
msgstr ""
#: models.py:474
#, fuzzy
#| msgid "Queued task"
msgid "Queue options"
msgstr "Sıraya alınmış iş"
#: models.py:478
msgid "Queued task"
msgstr "Sıraya alınmış iş"
#: django_q/models.py:263
#: models.py:479
msgid "Queued tasks"
msgstr "Sıraya alınmış işler"
#: django_q/monitor.py:62 django_q/monitor.py:339
#: monitor.py:41
#, python-format
msgid "%(name)s monitoring at %(id)s"
msgstr "%(name)s, %(id)s pid'inde izleniyor/monitoring yapılıyor."
#: monitor.py:61
#, python-format
msgid "Processed '%(info_name)s' (%(task_name)s)"
msgstr "[%(task_name)s] - '%(info_name)s işlendi."
#: monitor.py:67
#, python-format
msgid "Failed '%(info_name)s' (%(task_name)s) - %(task_result)s"
msgstr "[%(task_name)s] - '%(info_name)s' - %(task_result)s başarısız oldu"
#: monitor.py:74
#, python-format
msgid "%(name)s stopped monitoring results"
msgstr "%(name)s sonuçları göstermeyi bıraktı"
#: monitor_terminal.py:64 monitor_terminal.py:348
msgid "Host"
msgstr ""
#: django_q/monitor.py:66 django_q/monitor.py:343 django_q/monitor.py:450
#: monitor_terminal.py:68 monitor_terminal.py:352 monitor_terminal.py:457
msgid "Id"
msgstr ""
#: django_q/monitor.py:70
#: monitor_terminal.py:72
msgid "State"
msgstr "Durum"
#: django_q/monitor.py:74
#: monitor_terminal.py:76
msgid "Pool"
msgstr "Havuz"
#: django_q/monitor.py:78
#: monitor_terminal.py:80
msgid "TQ"
msgstr ""
#: django_q/monitor.py:82
#: monitor_terminal.py:84
msgid "RQ"
msgstr ""
#: django_q/monitor.py:86
#: monitor_terminal.py:88
msgid "RC"
msgstr ""
#: django_q/monitor.py:90
#: monitor_terminal.py:92
msgid "Up"
msgstr ""
#: django_q/monitor.py:170 django_q/monitor.py:279
#: monitor_terminal.py:172 monitor_terminal.py:286
msgid "Queued"
msgstr "Sıraya alınmış"
#: django_q/monitor.py:178
msgid "Success"
msgstr "Başarılı olanlar"
#: django_q/monitor.py:188 django_q/monitor.py:287
#: monitor_terminal.py:190 monitor_terminal.py:294
msgid "Failures"
msgstr "Başarısız olanlar"
#: django_q/monitor.py:199 django_q/monitor.py:485
#: monitor_terminal.py:201 monitor_terminal.py:496
msgid "[Press q to quit]"
msgstr "[Çıkmak için q'ya basın]"
#: django_q/monitor.py:223
#: monitor_terminal.py:227
msgid "day"
msgstr "gün"
#: django_q/monitor.py:244
#: monitor_terminal.py:248
msgid "second"
msgstr "saniye"
#: django_q/monitor.py:247
#: monitor_terminal.py:251
msgid "minute"
msgstr "dakika"
#: django_q/monitor.py:250
#: monitor_terminal.py:254
msgid "hour"
msgstr "saat"
#: django_q/monitor.py:260
#: monitor_terminal.py:263
#, python-format
msgid "-- %(prefix)s %(version)s on %(info)s --"
msgstr "-- %(prefix)s %(version)s üzerinde %(info)s --"
#: django_q/monitor.py:266
#: monitor_terminal.py:273
msgid "Clusters"
msgstr ""
#: django_q/monitor.py:270
#: monitor_terminal.py:277
msgid "Workers"
msgstr ""
#: django_q/monitor.py:274
#: monitor_terminal.py:281
msgid "Restarts"
msgstr "Yeniden çalıştırmalar"
#: django_q/monitor.py:283
#: monitor_terminal.py:290
msgid "Successes"
msgstr "Başarılı olanlar"
#: django_q/monitor.py:292
#: monitor_terminal.py:299
msgid "Schedules"
msgstr "Zamanlanmışlar"
#: django_q/monitor.py:296
#: monitor_terminal.py:303
#, python-format
msgid "Tasks/%(per)s"
msgstr "İş/%(per)s"
#: django_q/monitor.py:300
#: monitor_terminal.py:307
msgid "Avg time"
msgstr "Ortalama süre"
#: django_q/monitor.py:348
#: monitor_terminal.py:357
msgid "Available (%)"
msgstr "Müsait (%) "
#: django_q/monitor.py:354
#: monitor_terminal.py:363
msgid "Available (MB)"
msgstr "Müsait (MB)"
#: django_q/monitor.py:359
#: monitor_terminal.py:368
msgid "Total (MB)"
msgstr "Toplam (MB)"
#: django_q/monitor.py:364
#: monitor_terminal.py:373
msgid "Sentinel (MB)"
msgstr ""
#: django_q/monitor.py:370
#: monitor_terminal.py:379
msgid "Monitor (MB)"
msgstr "İzleme (MB)"
#: django_q/monitor.py:376
#: monitor_terminal.py:385
msgid "Workers (MB)"
msgstr ""
#: django_q/monitor.py:478
#: monitor_terminal.py:485
#, python-format
msgid "Available lowest (): %(memory_percent)s ((at)s)"
msgstr "Mevcut en düşük (): %(memory_percent)s ((at)s)"
#: django_q/monitor.py:496
#: monitor_terminal.py:507
msgid "No clusters appear to be running."
msgstr "Hiçbir küme çalışıyor görünmüyor."
#: django_q/signals.py:22
#: pusher.py:40
#, python-format
msgid "%(name)s pushing tasks at %(id)s"
msgstr "%(name)s, işleri %(id)s pid'ine gönderiyor."
#: pusher.py:67
#, python-format
msgid "queueing from %(list_key)s"
msgstr "%(list_key)s kuyruğa alınıyor"
#: pusher.py:71
#, python-format
msgid "%(name)s stopped pushing tasks"
msgstr "%(name)s işleri göndermeyi durdurdu"
#: scheduler.py:110
#, python-format
msgid "%(process_name)s failed to create a task from schedule [%(schedule)s]"
msgstr "%(process_name)s programdan bir görev oluşturamadı [%(schedule)s]"
#: scheduler.py:121
#, python-format
msgid ""
"%(process_name)s created task %(task_name)s from schedule [%(schedule)s]"
msgstr "%(process_name)s programdan %(task_name)s görevini oluşturdu [%(schedule)s]"
#: signals.py:22
#, python-format
msgid "malformed return hook '%(hook)s' for [%(name)s]"
msgstr ""
#: django_q/signals.py:30
#: signals.py:30
#, python-format
msgid "return hook %(hook)s failed on [%(name)s] because %(error)s"
msgstr ""
#: timeout.py:30 timeout.py:43
msgid "SIGALARM is not available on your platform"
msgstr ""
#: worker.py:49
#, python-format
msgid "%(proc_name)s ready for work at %(id)s"
msgstr "%(proc_name)s, %(id)s pid'inde çalışmaya hazır"
#: worker.py:69
#, fuzzy, python-format
#| msgid "%(proc_name)s processing '%(func_name)s' (%(task_name)s)"
msgid "%(proc_name)s processing %(task_name)s '%(func_name)s'"
msgstr "%(proc_name)s, '%(func_name)s' [%(task_name)s] işlerini işiyor"
#: worker.py:135
#, python-format
msgid "%(proc_name)s stopped doing work"
msgstr "%(proc_name)s çalışmayı bıraktı"
#, python-format
#~ msgid ""
#~ "Could not process '%(func_name)s'. Check the location of the function and "
#~ "the args/kwargs."
#~ msgstr ""
#~ "%(func_name)s' işlenemedi. İşlevin konumunu ve args/kwargs öğelerini "
#~ "kontrol edin."

View File

@@ -0,0 +1,682 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
msgid ""
msgstr ""
"Project-Id-Version: django-q\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-04-17 21:10+0000\n"
"PO-Revision-Date: 2026-01-14 16:36+0800\n"
"Last-Translator: Yibo Li <lybcyd@outlook.com>\n"
"Language-Team: Simplified Chinese\n"
"Language: zh_Hans\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Generator: \n"
"Plural-Forms: nplurals=1; plural=0;\n"
#: admin.py:30
msgid "Resubmit selected tasks to queue"
msgstr "将选定的任务重新提交到队列"
#: admin.py:64
msgid "Time Taken"
msgstr ""
#: admin.py:95
msgid "Short result"
msgstr ""
#: admin.py:142 models.py:412
#, fuzzy
#| msgid "Success"
msgid "success"
msgstr "成功"
#: admin.py:154 models.py:414
msgid "last_run"
msgstr "最后运行"
#: cluster.py:75
#, python-format
msgid "Q Cluster %(name)s starting."
msgstr "Q 集群 %(name)s 正在启动。"
#: cluster.py:83
#, fuzzy, python-format
#| msgid "Q Cluster-{} stopping."
msgid "Q Cluster %(name)s stopping."
msgstr "Q 集群 %(name)s 正在停止。"
#: cluster.py:86
#, python-format
msgid "Q Cluster %(name)s has stopped."
msgstr "Q 集群 %(name)s 已停止。"
#: cluster.py:93
#, python-format
msgid "%(name)s got signal %(signal)s"
msgstr "%(name)s 收到信号 %(signal)s"
#: cluster.py:224
#, python-format
msgid "reincarnated monitor %(name)s after sudden death"
msgstr "意外宕机后monitor %(name)s 已重生"
#: cluster.py:230
#, python-format
msgid "reincarnated pusher %(name)s after sudden death"
msgstr "意外宕机后pusher %(name)s 已重生"
#: cluster.py:262
#, fuzzy, python-format
#| msgid "reincarnated worker %(name)s after timeout"
msgid ""
"reincarnated worker %(name)s after timeout while processing task "
"%(task_name)s"
msgstr "处理任务 %(task_name)s 时超时worker %(name)s 已重生"
#: cluster.py:265
#, python-format
msgid "reincarnated worker %(name)s after timeout"
msgstr "超时后worker %(name)s 已重生"
#: cluster.py:270
#, python-format
msgid "recycled worker %(name)s"
msgstr "worker %(name)s 已回收"
#: cluster.py:273
#, python-format
msgid "reincarnated worker %(name)s after death"
msgstr "意外宕机后worker %(name)s 已重生"
#: cluster.py:297
#, python-format
msgid "%(name)s guarding cluster %(cluster_name)s"
msgstr "%(name)s 正在保护Cluster %(cluster_name)s"
#: cluster.py:307
#, python-format
msgid "Q Cluster %(cluster_name)s running."
msgstr "Q 集群 %(cluster_name)s 正在运行。"
#: cluster.py:346
#, python-format
msgid "%(name)s stopping cluster processes"
msgstr "%(name)s 正在停止集群进程"
#: cluster.py:371
#, python-format
msgid "%(name)s waiting for the monitor."
msgstr "%(name)s 正在等待监视器。"
#: cluster.py:394
msgid "Skipping cpu affinity because psutil was not found."
msgstr "跳过 CPU 亲和性,因为未找到 psutil。"
#: cluster.py:399
msgid "Faking cpu affinity because it is not supported on this platform"
msgstr "伪造 CPU 亲和性,因为此平台不支持此功能"
#: cluster.py:421
#, python-format
msgid "%(pid)s will use cpu %(affinity)s"
msgstr "%(pid)s 将使用 CPU %(affinity)s"
#: conf.py:122
#, python-format
msgid ""
"SAVE_LIMIT_PER (%(option)s) is not a valid option. Options are: 'group', "
"'name', 'func' and None. Default is None."
msgstr ""
"SAVE_LIMIT_PER (%(option)s) 不是有效的选项。选项包"
"括:'group'、'name'、'func' 和 None。默认值是 None。"
#. Translators: Cluster status descriptions
#: conf.py:239
msgid "Starting"
msgstr "正在启动"
#: conf.py:240
msgid "Working"
msgstr "工作中"
#: conf.py:241
msgid "Idle"
msgstr "空闲"
#: conf.py:242
msgid "Stopped"
msgstr "已停止"
#: conf.py:243
msgid "Stopping"
msgstr "正在停止"
#. Translators: help text for qcluster management command
#: management/commands/qcluster.py:11
msgid "Starts a Django Q Cluster."
msgstr "启动 Django Q 集群。"
#. Translators: help text for qinfo management command
#: management/commands/qinfo.py:11
msgid "General information over all clusters."
msgstr "所有集群的常规信息。"
#. Translators: help text for qmemory management command
#: management/commands/qmemory.py:9
msgid "Monitors Q Cluster memory usage"
msgstr "监视 Q 集群内存使用情况"
#. Translators: help text for qmonitor management command
#: management/commands/qmonitor.py:9
msgid "Monitors Q Cluster activity"
msgstr "监视 Q 集群活动"
#: models.py:30 models.py:469
msgid "Task id"
msgstr ""
#: models.py:31
msgid "32-character task identifier."
msgstr ""
#: models.py:36 models.py:235 models.py:470
msgid "Name"
msgstr ""
#: models.py:37
msgid "Optional human-readable name for lookup and display."
msgstr ""
#: models.py:41 models.py:240 models.py:468
msgid "Function"
msgstr ""
#: models.py:42
msgid "Dotted import path to the callable, e.g. myapp.tasks.job."
msgstr ""
#: models.py:47 models.py:247
msgid "Hook"
msgstr ""
#: models.py:48
msgid "Optional dotted path to a callback that receives the task result."
msgstr ""
#: models.py:53 models.py:253 models.py:472
msgid "Arguments"
msgstr ""
#: models.py:54
msgid "Pickled positional arguments (tuple)."
msgstr ""
#: models.py:59 models.py:259 models.py:473
msgid "Keyword arguments"
msgstr ""
#: models.py:60
msgid "Pickled keyword arguments (dict)."
msgstr ""
#: models.py:65
msgid "Result"
msgstr ""
#: models.py:66
msgid "Pickled return value or error payload."
msgstr ""
#: models.py:72 models.py:471
msgid "Group"
msgstr ""
#: models.py:73
msgid "Optional group id to aggregate related tasks."
msgstr ""
#: models.py:80 models.py:328
#, fuzzy
#| msgid "Clusters"
msgid "Cluster"
msgstr "集群"
#: models.py:81
msgid "Name of the cluster that executed this task."
msgstr ""
#: models.py:85
msgid "Started at"
msgstr ""
#: models.py:86
msgid "When the worker started executing the task."
msgstr ""
#: models.py:90
#, fuzzy
#| msgid "Stopped"
msgid "Stopped at"
msgstr "已停止"
#: models.py:91
msgid "When the worker finished (success or failure)."
msgstr ""
#: models.py:96 monitor_terminal.py:180
msgid "Success"
msgstr "成功"
#: models.py:97
msgid "False if the task raised an exception or timed out."
msgstr ""
#: models.py:101
msgid "Attempt count"
msgstr ""
#: models.py:102
msgid "How many times execution was attempted."
msgstr ""
#: models.py:172
msgid "Task"
msgstr ""
#: models.py:173
msgid "Tasks"
msgstr ""
#: models.py:194
msgid "Successful task"
msgstr "成功的任务"
#: models.py:195
msgid "Successful tasks"
msgstr "成功的任务"
#: models.py:210
msgid "Failed task"
msgstr "失败的任务"
#: models.py:211
msgid "Failed tasks"
msgstr "失败的任务"
#: models.py:219 models.py:347
msgid "Please install croniter to enable cron expressions"
msgstr "请安装 croniter 以启用 cron 表达式"
#: models.py:236
msgid "Optional label to identify this schedule in the admin."
msgstr ""
#: models.py:241
msgid "e.g. module.tasks.function"
msgstr ""
#: models.py:248
msgid "e.g. module.tasks.result_function"
msgstr ""
#: models.py:254
msgid "e.g. 1, 2, 'John'"
msgstr "例如 1, 2, 'John'"
#: models.py:260
msgid "e.g. x=1, y=2, name='John'"
msgstr "例如 x=1, y=2, name='John'"
#: models.py:274
msgid "Once"
msgstr "一次"
#: models.py:275 models.py:296
msgid "Minutes"
msgstr "分钟"
#: models.py:276
msgid "Hourly"
msgstr "每小时"
#: models.py:277
msgid "Daily"
msgstr "每天"
#: models.py:278
msgid "Weekly"
msgstr "每周"
#: models.py:279
msgid "Biweekly"
msgstr "每两周"
#: models.py:280
msgid "Monthly"
msgstr "每月"
#: models.py:281
msgid "Bimonthly"
msgstr "每两个月"
#: models.py:282
msgid "Quarterly"
msgstr "每季度"
#: models.py:283
msgid "Yearly"
msgstr "每年"
#: models.py:284 models.py:313
msgid "Cron"
msgstr "Cron"
#: models.py:290
msgid "Schedule Type"
msgstr "计划类型"
#: models.py:291
msgid "How often this task should be enqueued."
msgstr ""
#: models.py:297
msgid "Number of minutes for the Minutes type"
msgstr "'分钟'类型的分钟数"
#: models.py:300
msgid "Repeats"
msgstr "重复次数"
#: models.py:300
msgid "n = n times, -1 = forever"
msgstr "n = n 次,-1 = 永远"
#: models.py:303
msgid "Next Run"
msgstr "下次运行"
#: models.py:304
msgid "When this schedule runs next (stored in UTC)."
msgstr ""
#: models.py:314
msgid "Cron expression"
msgstr "Cron 表达式"
#: models.py:320
msgid "Last task id"
msgstr ""
#: models.py:321
msgid "Id of the last task spawned from this schedule (read-only)."
msgstr ""
#: models.py:329 models.py:427
msgid "Name of the target cluster"
msgstr ""
#: models.py:336
msgid "Intended date kwarg"
msgstr ""
#: models.py:337
msgid "Name of kwarg to pass intended schedule date"
msgstr "传递预定计划日期的 kwarg 的名称"
#: models.py:418
msgid "Scheduled task"
msgstr "计划任务"
#: models.py:419
msgid "Scheduled tasks"
msgstr "计划任务"
#: models.py:426
#, fuzzy
#| msgid "Clusters"
msgid "Cluster key"
msgstr "集群"
#: models.py:430
msgid "Payload"
msgstr ""
#: models.py:431
msgid "Signed serialized task package (do not edit manually)."
msgstr ""
#: models.py:435
msgid "Lock until"
msgstr ""
#: models.py:436
msgid "Prevent any cluster from pulling until"
msgstr ""
#: models.py:474
#, fuzzy
#| msgid "Queued task"
msgid "Queue options"
msgstr "排队的任务"
#: models.py:478
msgid "Queued task"
msgstr "排队的任务"
#: models.py:479
msgid "Queued tasks"
msgstr "排队的任务"
#: monitor.py:41
#, python-format
msgid "%(name)s monitoring at %(id)s"
msgstr "%(name)s 正在 %(id)s 处监视"
#: monitor.py:61
#, python-format
msgid "Processed '%(info_name)s' (%(task_name)s)"
msgstr "已处理 '%(info_name)s' (%(task_name)s)"
#: monitor.py:67
#, python-format
msgid "Failed '%(info_name)s' (%(task_name)s) - %(task_result)s"
msgstr "'%(info_name)s' (%(task_name)s) 失败 - %(task_result)s"
#: monitor.py:74
#, python-format
msgid "%(name)s stopped monitoring results"
msgstr "%(name)s 已停止监视结果"
#: monitor_terminal.py:64 monitor_terminal.py:348
msgid "Host"
msgstr "主机"
#: monitor_terminal.py:68 monitor_terminal.py:352 monitor_terminal.py:457
msgid "Id"
msgstr "ID"
#: monitor_terminal.py:72
msgid "State"
msgstr "状态"
#: monitor_terminal.py:76
msgid "Pool"
msgstr "连接池"
#: monitor_terminal.py:80
msgid "TQ"
msgstr "TQ"
#: monitor_terminal.py:84
msgid "RQ"
msgstr "RQ"
#: monitor_terminal.py:88
msgid "RC"
msgstr "RC"
#: monitor_terminal.py:92
msgid "Up"
msgstr "运行中"
#: monitor_terminal.py:172 monitor_terminal.py:286
msgid "Queued"
msgstr "已排队"
#: monitor_terminal.py:190 monitor_terminal.py:294
msgid "Failures"
msgstr "失败"
#: monitor_terminal.py:201 monitor_terminal.py:496
msgid "[Press q to quit]"
msgstr "[按 q 键退出]"
#: monitor_terminal.py:227
msgid "day"
msgstr "天"
#: monitor_terminal.py:248
msgid "second"
msgstr "秒"
#: monitor_terminal.py:251
msgid "minute"
msgstr "分钟"
#: monitor_terminal.py:254
msgid "hour"
msgstr "小时"
#: monitor_terminal.py:263
#, python-format
msgid "-- %(prefix)s %(version)s on %(info)s --"
msgstr "-- %(prefix)s %(version)s 在 %(info)s 上 --"
#: monitor_terminal.py:273
msgid "Clusters"
msgstr "集群"
#: monitor_terminal.py:277
msgid "Workers"
msgstr "工作进程"
#: monitor_terminal.py:281
msgid "Restarts"
msgstr "重启次数"
#: monitor_terminal.py:290
msgid "Successes"
msgstr "成功次数"
#: monitor_terminal.py:299
msgid "Schedules"
msgstr "计划"
#: monitor_terminal.py:303
#, python-format
msgid "Tasks/%(per)s"
msgstr "任务/%(per)s"
#: monitor_terminal.py:307
msgid "Avg time"
msgstr "平均时间"
#: monitor_terminal.py:357
msgid "Available (%)"
msgstr "可用 (%)"
#: monitor_terminal.py:363
msgid "Available (MB)"
msgstr "可用 (MB)"
#: monitor_terminal.py:368
msgid "Total (MB)"
msgstr "总计 (MB)"
#: monitor_terminal.py:373
msgid "Sentinel (MB)"
msgstr "哨兵 (MB)"
#: monitor_terminal.py:379
msgid "Monitor (MB)"
msgstr "监视器 (MB)"
#: monitor_terminal.py:385
msgid "Workers (MB)"
msgstr "工作进程 (MB)"
#: monitor_terminal.py:485
#, python-format
msgid "Available lowest (): %(memory_percent)s ((at)s)"
msgstr "最低可用 (): %(memory_percent)s ((at)s)"
#: monitor_terminal.py:507
msgid "No clusters appear to be running."
msgstr "似乎没有集群正在运行。"
#: pusher.py:40
#, python-format
msgid "%(name)s pushing tasks at %(id)s"
msgstr "%(name)s 正在 %(id)s 处推送任务"
#: pusher.py:67
#, python-format
msgid "queueing from %(list_key)s"
msgstr "从 %(list_key)s 排队"
#: pusher.py:71
#, python-format
msgid "%(name)s stopped pushing tasks"
msgstr "%(name)s 已停止推送任务"
#: scheduler.py:110
#, python-format
msgid "%(process_name)s failed to create a task from schedule [%(schedule)s]"
msgstr "%(process_name)s 无法从计划 [%(schedule)s] 创建任务"
#: scheduler.py:121
#, python-format
msgid ""
"%(process_name)s created task %(task_name)s from schedule [%(schedule)s]"
msgstr "%(process_name)s 从计划 [%(schedule)s] 创建了任务 %(task_name)s"
#: signals.py:22
#, python-format
msgid "malformed return hook '%(hook)s' for [%(name)s]"
msgstr "格式错误的返回钩子 '%(hook)s' 用于 [%(name)s]"
#: signals.py:30
#, python-format
msgid "return hook %(hook)s failed on [%(name)s] because %(error)s"
msgstr "返回钩子 %(hook)s 在 [%(name)s] 上失败,原因是 %(error)s"
#: timeout.py:30 timeout.py:43
msgid "SIGALARM is not available on your platform"
msgstr ""
#: worker.py:49
#, python-format
msgid "%(proc_name)s ready for work at %(id)s"
msgstr "%(proc_name)s 已准备好在 %(id)s 处工作"
#: worker.py:69
#, fuzzy, python-format
#| msgid "%(proc_name)s processing '%(func_name)s' (%(task_name)s)"
msgid "%(proc_name)s processing %(task_name)s '%(func_name)s'"
msgstr "%(proc_name)s 正在处理任务 %(task_name)s '%(func_name)s'"
#: worker.py:135
#, python-format
msgid "%(proc_name)s stopped doing work"
msgstr "%(proc_name)s 已停止工作"
#, python-format
#~ msgid ""
#~ "Could not process '%(func_name)s'. Check the location of the function and "
#~ "the args/kwargs."
#~ msgstr "无法处理函数 '%(func_name)s'。检查函数的位置和args/kwargs。"

View File

@@ -1,3 +1,5 @@
import os
from django.core.management.base import BaseCommand
from django.utils.translation import gettext as _
@@ -16,8 +18,21 @@ class Command(BaseCommand):
default=False,
help="Run once and then stop.",
)
parser.add_argument(
"-n",
"--name",
dest="cluster_name",
default=None,
help="Set alternative cluster name instead of the name in Q_CLUSTER settings (for multi-queue setup). "
"On Linux you should set name through `Q_CLUSTER_NAME=cluster_name python manage.py qcluster` instead.",
)
def handle(self, *args, **options):
# Set alternative cluster_name before creating the cluster (cluster_name is broker's queue_name, too)
cluster_name = options.get("cluster_name")
if cluster_name:
os.environ["Q_CLUSTER_NAME"] = cluster_name
q = Cluster()
q.start()
if options.get("run_once", False):

View File

@@ -3,7 +3,7 @@ from django.utils.translation import gettext as _
from django_q import VERSION
from django_q.conf import Conf
from django_q.monitor import get_ids, info
from django_q.monitor_terminal import get_ids, info
class Command(BaseCommand):

View File

@@ -1,7 +1,7 @@
from django.core.management.base import BaseCommand
from django.utils.translation import gettext as _
from django_q.monitor import memory
from django_q.monitor_terminal import memory
class Command(BaseCommand):

View File

@@ -1,7 +1,7 @@
from django.core.management.base import BaseCommand
from django.utils.translation import gettext as _
from django_q.monitor import monitor
from django_q.monitor_terminal import monitor
class Command(BaseCommand):

View File

@@ -4,62 +4,142 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
]
dependencies = []
operations = [
migrations.CreateModel(
name='Schedule',
name="Schedule",
fields=[
('id', models.AutoField(verbose_name='ID', auto_created=True, serialize=False, primary_key=True)),
('func', models.CharField(max_length=256, help_text='e.g. module.tasks.function')),
('hook', models.CharField(null=True, blank=True, max_length=256, help_text='e.g. module.tasks.result_function')),
('args', models.CharField(null=True, blank=True, max_length=256, help_text="e.g. 1, 2, 'John'")),
('kwargs', models.CharField(null=True, blank=True, max_length=256, help_text="e.g. x=1, y=2, name='John'")),
('schedule_type', models.CharField(verbose_name='Schedule Type', choices=[('O', 'Once'), ('H', 'Hourly'), ('D', 'Daily'), ('W', 'Weekly'), ('M', 'Monthly'), ('Q', 'Quarterly'), ('Y', 'Yearly')], default='O', max_length=1)),
('repeats', models.SmallIntegerField(verbose_name='Repeats', default=-1, help_text='n = n times, -1 = forever')),
('next_run', models.DateTimeField(verbose_name='Next Run', default=django.utils.timezone.now, null=True)),
('task', models.CharField(editable=False, null=True, max_length=100)),
(
"id",
models.AutoField(
verbose_name="ID",
auto_created=True,
serialize=False,
primary_key=True,
),
),
(
"func",
models.CharField(
max_length=256, help_text="e.g. module.tasks.function"
),
),
(
"hook",
models.CharField(
null=True,
blank=True,
max_length=256,
help_text="e.g. module.tasks.result_function",
),
),
(
"args",
models.CharField(
null=True,
blank=True,
max_length=256,
help_text="e.g. 1, 2, 'John'",
),
),
(
"kwargs",
models.CharField(
null=True,
blank=True,
max_length=256,
help_text="e.g. x=1, y=2, name='John'",
),
),
(
"schedule_type",
models.CharField(
verbose_name="Schedule Type",
choices=[
("O", "Once"),
("H", "Hourly"),
("D", "Daily"),
("W", "Weekly"),
("M", "Monthly"),
("Q", "Quarterly"),
("Y", "Yearly"),
],
default="O",
max_length=1,
),
),
(
"repeats",
models.SmallIntegerField(
verbose_name="Repeats",
default=-1,
help_text="n = n times, -1 = forever",
),
),
(
"next_run",
models.DateTimeField(
verbose_name="Next Run",
default=django.utils.timezone.now,
null=True,
),
),
("task", models.CharField(editable=False, null=True, max_length=100)),
],
options={
'verbose_name': 'Scheduled task',
'ordering': ['next_run'],
"verbose_name": "Scheduled task",
"ordering": ["next_run"],
},
),
migrations.CreateModel(
name='Task',
name="Task",
fields=[
('id', models.AutoField(verbose_name='ID', auto_created=True, serialize=False, primary_key=True)),
('name', models.CharField(editable=False, max_length=100)),
('func', models.CharField(max_length=256)),
('hook', models.CharField(null=True, max_length=256)),
('args', picklefield.fields.PickledObjectField(editable=False, null=True)),
('kwargs', picklefield.fields.PickledObjectField(editable=False, null=True)),
('result', picklefield.fields.PickledObjectField(editable=False, null=True)),
('started', models.DateTimeField(editable=False)),
('stopped', models.DateTimeField(editable=False)),
('success', models.BooleanField(editable=False, default=True)),
(
"id",
models.AutoField(
verbose_name="ID",
auto_created=True,
serialize=False,
primary_key=True,
),
),
("name", models.CharField(editable=False, max_length=100)),
("func", models.CharField(max_length=256)),
("hook", models.CharField(null=True, max_length=256)),
(
"args",
picklefield.fields.PickledObjectField(editable=False, null=True),
),
(
"kwargs",
picklefield.fields.PickledObjectField(editable=False, null=True),
),
(
"result",
picklefield.fields.PickledObjectField(editable=False, null=True),
),
("started", models.DateTimeField(editable=False)),
("stopped", models.DateTimeField(editable=False)),
("success", models.BooleanField(editable=False, default=True)),
],
),
migrations.CreateModel(
name='Failure',
fields=[
],
name="Failure",
fields=[],
options={
'verbose_name': 'Failed task',
'proxy': True,
"verbose_name": "Failed task",
"proxy": True,
},
bases=('django_q.task',),
bases=("django_q.task",),
),
migrations.CreateModel(
name='Success',
fields=[
],
name="Success",
fields=[],
options={
'verbose_name': 'Successful task',
'proxy': True,
"verbose_name": "Successful task",
"proxy": True,
},
bases=('django_q.task',),
bases=("django_q.task",),
),
]

View File

@@ -2,20 +2,23 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0001_initial'),
("django_q", "0001_initial"),
]
operations = [
migrations.AlterField(
model_name='schedule',
name='args',
field=models.TextField(help_text="e.g. 1, 2, 'John'", blank=True, null=True),
model_name="schedule",
name="args",
field=models.TextField(
help_text="e.g. 1, 2, 'John'", blank=True, null=True
),
),
migrations.AlterField(
model_name='schedule',
name='kwargs',
field=models.TextField(help_text="e.g. x=1, y=2, name='John'", blank=True, null=True),
model_name="schedule",
name="kwargs",
field=models.TextField(
help_text="e.g. x=1, y=2, name='John'", blank=True, null=True
),
),
]

View File

@@ -2,31 +2,42 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0002_auto_20150630_1624'),
("django_q", "0002_auto_20150630_1624"),
]
operations = [
migrations.AlterModelOptions(
name='failure',
options={'verbose_name_plural': 'Failed tasks', 'verbose_name': 'Failed task'},
name="failure",
options={
"verbose_name_plural": "Failed tasks",
"verbose_name": "Failed task",
},
),
migrations.AlterModelOptions(
name='schedule',
options={'verbose_name_plural': 'Scheduled tasks', 'ordering': ['next_run'], 'verbose_name': 'Scheduled task'},
name="schedule",
options={
"verbose_name_plural": "Scheduled tasks",
"ordering": ["next_run"],
"verbose_name": "Scheduled task",
},
),
migrations.AlterModelOptions(
name='success',
options={'verbose_name_plural': 'Successful tasks', 'verbose_name': 'Successful task'},
name="success",
options={
"verbose_name_plural": "Successful tasks",
"verbose_name": "Successful task",
},
),
migrations.RemoveField(
model_name='task',
name='id',
model_name="task",
name="id",
),
migrations.AddField(
model_name='task',
name='id',
field=models.CharField(max_length=32, primary_key=True, editable=False, serialize=False),
model_name="task",
name="id",
field=models.CharField(
max_length=32, primary_key=True, editable=False, serialize=False
),
),
]

View File

@@ -1,23 +1,30 @@
from django.db import migrations, models
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('django_q', '0003_auto_20150708_1326'),
("django_q", "0003_auto_20150708_1326"),
]
operations = [
migrations.AlterModelOptions(
name='failure',
options={'verbose_name_plural': 'Failed tasks', 'verbose_name': 'Failed task', 'ordering': ['-stopped']},
name="failure",
options={
"verbose_name_plural": "Failed tasks",
"verbose_name": "Failed task",
"ordering": ["-stopped"],
},
),
migrations.AlterModelOptions(
name='success',
options={'verbose_name_plural': 'Successful tasks', 'verbose_name': 'Successful task', 'ordering': ['-stopped']},
name="success",
options={
"verbose_name_plural": "Successful tasks",
"verbose_name": "Successful task",
"ordering": ["-stopped"],
},
),
migrations.AlterModelOptions(
name='task',
options={'ordering': ['-stopped']},
name="task",
options={"ordering": ["-stopped"]},
),
]

View File

@@ -2,20 +2,19 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0004_auto_20150710_1043'),
("django_q", "0004_auto_20150710_1043"),
]
operations = [
migrations.AddField(
model_name='schedule',
name='name',
model_name="schedule",
name="name",
field=models.CharField(max_length=100, null=True),
),
migrations.AddField(
model_name='task',
name='group',
model_name="task",
name="group",
field=models.CharField(max_length=100, null=True, editable=False),
),
]

View File

@@ -2,20 +2,37 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0005_auto_20150718_1506'),
("django_q", "0005_auto_20150718_1506"),
]
operations = [
migrations.AddField(
model_name='schedule',
name='minutes',
field=models.PositiveSmallIntegerField(help_text='Number of minutes for the Minutes type', blank=True, null=True),
model_name="schedule",
name="minutes",
field=models.PositiveSmallIntegerField(
help_text="Number of minutes for the Minutes type",
blank=True,
null=True,
),
),
migrations.AlterField(
model_name='schedule',
name='schedule_type',
field=models.CharField(max_length=1, choices=[('O', 'Once'), ('I', 'Minutes'), ('H', 'Hourly'), ('D', 'Daily'), ('W', 'Weekly'), ('M', 'Monthly'), ('Q', 'Quarterly'), ('Y', 'Yearly')], default='O', verbose_name='Schedule Type'),
model_name="schedule",
name="schedule_type",
field=models.CharField(
max_length=1,
choices=[
("O", "Once"),
("I", "Minutes"),
("H", "Hourly"),
("D", "Daily"),
("W", "Weekly"),
("M", "Monthly"),
("Q", "Quarterly"),
("Y", "Yearly"),
],
default="O",
verbose_name="Schedule Type",
),
),
]

View File

@@ -2,23 +2,30 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0006_auto_20150805_1817'),
("django_q", "0006_auto_20150805_1817"),
]
operations = [
migrations.CreateModel(
name='OrmQ',
name="OrmQ",
fields=[
('id', models.AutoField(primary_key=True, auto_created=True, verbose_name='ID', serialize=False)),
('key', models.CharField(max_length=100)),
('payload', models.TextField()),
('lock', models.DateTimeField(null=True)),
(
"id",
models.AutoField(
primary_key=True,
auto_created=True,
verbose_name="ID",
serialize=False,
),
),
("key", models.CharField(max_length=100)),
("payload", models.TextField()),
("lock", models.DateTimeField(null=True)),
],
options={
'verbose_name_plural': 'Queued tasks',
'verbose_name': 'Queued task',
"verbose_name_plural": "Queued tasks",
"verbose_name": "Queued task",
},
),
]

View File

@@ -2,15 +2,14 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0007_ormq'),
("django_q", "0007_ormq"),
]
operations = [
migrations.AlterField(
model_name='schedule',
name='name',
model_name="schedule",
name="name",
field=models.CharField(blank=True, max_length=100, null=True),
),
]

View File

@@ -2,15 +2,18 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0008_auto_20160224_1026'),
("django_q", "0008_auto_20160224_1026"),
]
operations = [
migrations.AlterField(
model_name='schedule',
name='repeats',
field=models.IntegerField(default=-1, help_text='n = n times, -1 = forever', verbose_name='Repeats'),
model_name="schedule",
name="repeats",
field=models.IntegerField(
default=-1,
help_text="n = n times, -1 = forever",
verbose_name="Repeats",
),
),
]

View File

@@ -3,25 +3,30 @@ from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('django_q', '0009_auto_20171009_0915'),
("django_q", "0009_auto_20171009_0915"),
]
operations = [
migrations.AlterField(
model_name='task',
name='args',
field=picklefield.fields.PickledObjectField(editable=False, null=True, protocol=-1),
model_name="task",
name="args",
field=picklefield.fields.PickledObjectField(
editable=False, null=True, protocol=-1
),
),
migrations.AlterField(
model_name='task',
name='kwargs',
field=picklefield.fields.PickledObjectField(editable=False, null=True, protocol=-1),
model_name="task",
name="kwargs",
field=picklefield.fields.PickledObjectField(
editable=False, null=True, protocol=-1
),
),
migrations.AlterField(
model_name='task',
name='result',
field=picklefield.fields.PickledObjectField(editable=False, null=True, protocol=-1),
model_name="task",
name="result",
field=picklefield.fields.PickledObjectField(
editable=False, null=True, protocol=-1
),
),
]

View File

@@ -4,20 +4,36 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0010_auto_20200610_0856'),
("django_q", "0010_auto_20200610_0856"),
]
operations = [
migrations.AddField(
model_name='schedule',
name='cron',
field=models.CharField(blank=True, help_text='Cron expression', max_length=100, null=True),
model_name="schedule",
name="cron",
field=models.CharField(
blank=True, help_text="Cron expression", max_length=100, null=True
),
),
migrations.AlterField(
model_name='schedule',
name='schedule_type',
field=models.CharField(choices=[('O', 'Once'), ('I', 'Minutes'), ('H', 'Hourly'), ('D', 'Daily'), ('W', 'Weekly'), ('M', 'Monthly'), ('Q', 'Quarterly'), ('Y', 'Yearly'), ('C', 'Cron')], default='O', max_length=1, verbose_name='Schedule Type'),
model_name="schedule",
name="schedule_type",
field=models.CharField(
choices=[
("O", "Once"),
("I", "Minutes"),
("H", "Hourly"),
("D", "Daily"),
("W", "Weekly"),
("M", "Monthly"),
("Q", "Quarterly"),
("Y", "Yearly"),
("C", "Cron"),
],
default="O",
max_length=1,
verbose_name="Schedule Type",
),
),
]

View File

@@ -6,15 +6,20 @@ import django_q.models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0011_auto_20200628_1055'),
("django_q", "0011_auto_20200628_1055"),
]
operations = [
migrations.AlterField(
model_name='schedule',
name='cron',
field=models.CharField(blank=True, help_text='Cron expression', max_length=100, null=True, validators=[django_q.models.validate_cron]),
model_name="schedule",
name="cron",
field=models.CharField(
blank=True,
help_text="Cron expression",
max_length=100,
null=True,
validators=[django_q.models.validate_cron],
),
),
]

View File

@@ -4,15 +4,14 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0012_auto_20200702_1608'),
("django_q", "0012_auto_20200702_1608"),
]
operations = [
migrations.AddField(
model_name='task',
name='attempt_count',
model_name="task",
name="attempt_count",
field=models.IntegerField(default=0),
),
]

View File

@@ -4,15 +4,14 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0013_task_attempt_count'),
("django_q", "0013_task_attempt_count"),
]
operations = [
migrations.AddField(
model_name='schedule',
name='cluster',
model_name="schedule",
name="cluster",
field=models.CharField(blank=True, default=None, max_length=100, null=True),
),
]

View File

@@ -4,15 +4,31 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('django_q', '0014_schedule_cluster'),
("django_q", "0014_schedule_cluster"),
]
operations = [
migrations.AlterField(
model_name='schedule',
name='schedule_type',
field=models.CharField(choices=[('O', 'Once'), ('I', 'Minutes'), ('H', 'Hourly'), ('D', 'Daily'), ('W', 'Weekly'), ('BW', 'Biweekly'), ('M', 'Monthly'), ('BM', 'Bimonthly'), ('Q', 'Quarterly'), ('Y', 'Yearly'), ('C', 'Cron')], default='O', max_length=2, verbose_name='Schedule Type'),
model_name="schedule",
name="schedule_type",
field=models.CharField(
choices=[
("O", "Once"),
("I", "Minutes"),
("H", "Hourly"),
("D", "Daily"),
("W", "Weekly"),
("BW", "Biweekly"),
("M", "Monthly"),
("BM", "Bimonthly"),
("Q", "Quarterly"),
("Y", "Yearly"),
("C", "Cron"),
],
default="O",
max_length=2,
verbose_name="Schedule Type",
),
),
]

View File

@@ -0,0 +1,25 @@
# Generated by Django 4.1.2 on 2023-01-15 22:34
from django.db import migrations, models
import django_q.models
class Migration(migrations.Migration):
dependencies = [
("django_q", "0015_alter_schedule_schedule_type"),
]
operations = [
migrations.AddField(
model_name="schedule",
name="intended_date_kwarg",
field=models.CharField(
blank=True,
help_text="Name of kwarg to pass intended schedule date",
max_length=100,
null=True,
validators=[django_q.models.validate_kwarg],
),
),
]

View File

@@ -0,0 +1,42 @@
# Generated by Django 4.1.5 on 2023-03-07 12:18
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("django_q", "0016_schedule_intended_date_kwarg"),
]
operations = [
migrations.AddField(
model_name="task",
name="cluster",
field=models.CharField(blank=True, default=None, max_length=100, null=True),
),
migrations.AlterField(
model_name="ormq",
name="key",
field=models.CharField(
help_text="Name of the target cluster", max_length=100
),
),
migrations.AlterField(
model_name="ormq",
name="lock",
field=models.DateTimeField(
help_text="Prevent any cluster from pulling until", null=True
),
),
migrations.AlterField(
model_name="schedule",
name="cluster",
field=models.CharField(
blank=True,
default=None,
help_text="Name of the target cluster",
max_length=100,
null=True,
),
),
]

View File

@@ -0,0 +1,20 @@
# Generated by Django 4.2.7 on 2024-03-05 17:03
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("django_q", "0017_task_cluster_alter"),
]
operations = [
migrations.AddIndex(
model_name="task",
index=models.Index(
condition=models.Q(("success", True)),
fields=["group", "name", "func"],
name="success_index",
),
),
]

View File

@@ -0,0 +1,325 @@
# Generated by Django 6.0.4 on 2026-04-17 20:55
import django.utils.timezone
import picklefield.fields
from django.db import migrations, models
import django_q.models
class Migration(migrations.Migration):
dependencies = [
("django_q", "0018_task_success_index"),
]
operations = [
migrations.AlterModelOptions(
name="task",
options={
"ordering": ["-stopped"],
"verbose_name": "Task",
"verbose_name_plural": "Tasks",
},
),
migrations.AlterField(
model_name="ormq",
name="key",
field=models.CharField(
help_text="Name of the target cluster",
max_length=100,
verbose_name="Cluster key",
),
),
migrations.AlterField(
model_name="ormq",
name="lock",
field=models.DateTimeField(
help_text="Prevent any cluster from pulling until",
null=True,
verbose_name="Lock until",
),
),
migrations.AlterField(
model_name="ormq",
name="payload",
field=models.TextField(
help_text="Signed serialized task package (do not edit manually).",
verbose_name="Payload",
),
),
migrations.AlterField(
model_name="schedule",
name="args",
field=models.TextField(
blank=True,
help_text="e.g. 1, 2, 'John'",
null=True,
verbose_name="Arguments",
),
),
migrations.AlterField(
model_name="schedule",
name="cluster",
field=models.CharField(
blank=True,
default=None,
help_text="Name of the target cluster",
max_length=100,
null=True,
verbose_name="Cluster",
),
),
migrations.AlterField(
model_name="schedule",
name="cron",
field=models.CharField(
blank=True,
help_text="Cron expression",
max_length=100,
null=True,
validators=[django_q.models.validate_cron],
verbose_name="Cron",
),
),
migrations.AlterField(
model_name="schedule",
name="func",
field=models.CharField(
help_text="e.g. module.tasks.function",
max_length=256,
verbose_name="Function",
),
),
migrations.AlterField(
model_name="schedule",
name="hook",
field=models.CharField(
blank=True,
help_text="e.g. module.tasks.result_function",
max_length=256,
null=True,
verbose_name="Hook",
),
),
migrations.AlterField(
model_name="schedule",
name="intended_date_kwarg",
field=models.CharField(
blank=True,
help_text="Name of kwarg to pass intended schedule date",
max_length=100,
null=True,
validators=[django_q.models.validate_kwarg],
verbose_name="Intended date kwarg",
),
),
migrations.AlterField(
model_name="schedule",
name="kwargs",
field=models.TextField(
blank=True,
help_text="e.g. x=1, y=2, name='John'",
null=True,
verbose_name="Keyword arguments",
),
),
migrations.AlterField(
model_name="schedule",
name="minutes",
field=models.PositiveSmallIntegerField(
blank=True,
help_text="Number of minutes for the Minutes type",
null=True,
verbose_name="Minutes",
),
),
migrations.AlterField(
model_name="schedule",
name="name",
field=models.CharField(
blank=True,
help_text="Optional label to identify this schedule in the admin.",
max_length=100,
null=True,
verbose_name="Name",
),
),
migrations.AlterField(
model_name="schedule",
name="next_run",
field=models.DateTimeField(
default=django.utils.timezone.now,
help_text="When this schedule runs next (stored in UTC).",
null=True,
verbose_name="Next Run",
),
),
migrations.AlterField(
model_name="schedule",
name="schedule_type",
field=models.CharField(
choices=[
("O", "Once"),
("I", "Minutes"),
("H", "Hourly"),
("D", "Daily"),
("W", "Weekly"),
("BW", "Biweekly"),
("M", "Monthly"),
("BM", "Bimonthly"),
("Q", "Quarterly"),
("Y", "Yearly"),
("C", "Cron"),
],
default="O",
help_text="How often this task should be enqueued.",
max_length=2,
verbose_name="Schedule Type",
),
),
migrations.AlterField(
model_name="schedule",
name="task",
field=models.CharField(
editable=False,
help_text="Id of the last task spawned from this schedule (read-only).",
max_length=100,
null=True,
verbose_name="Last task id",
),
),
migrations.AlterField(
model_name="task",
name="args",
field=picklefield.fields.PickledObjectField(
editable=False,
help_text="Pickled positional arguments (tuple).",
null=True,
protocol=-1,
verbose_name="Arguments",
),
),
migrations.AlterField(
model_name="task",
name="attempt_count",
field=models.IntegerField(
default=0,
help_text="How many times execution was attempted.",
verbose_name="Attempt count",
),
),
migrations.AlterField(
model_name="task",
name="cluster",
field=models.CharField(
blank=True,
default=None,
help_text="Name of the cluster that executed this task.",
max_length=100,
null=True,
verbose_name="Cluster",
),
),
migrations.AlterField(
model_name="task",
name="func",
field=models.CharField(
help_text="Dotted import path to the callable, e.g. myapp.tasks.job.",
max_length=256,
verbose_name="Function",
),
),
migrations.AlterField(
model_name="task",
name="group",
field=models.CharField(
editable=False,
help_text="Optional group id to aggregate related tasks.",
max_length=100,
null=True,
verbose_name="Group",
),
),
migrations.AlterField(
model_name="task",
name="hook",
field=models.CharField(
help_text="Optional dotted path to a callback that receives the task result.",
max_length=256,
null=True,
verbose_name="Hook",
),
),
migrations.AlterField(
model_name="task",
name="id",
field=models.CharField(
editable=False,
help_text="32-character task identifier.",
max_length=32,
primary_key=True,
serialize=False,
verbose_name="Task id",
),
),
migrations.AlterField(
model_name="task",
name="kwargs",
field=picklefield.fields.PickledObjectField(
editable=False,
help_text="Pickled keyword arguments (dict).",
null=True,
protocol=-1,
verbose_name="Keyword arguments",
),
),
migrations.AlterField(
model_name="task",
name="name",
field=models.CharField(
editable=False,
help_text="Optional human-readable name for lookup and display.",
max_length=100,
verbose_name="Name",
),
),
migrations.AlterField(
model_name="task",
name="result",
field=picklefield.fields.PickledObjectField(
editable=False,
help_text="Pickled return value or error payload.",
null=True,
protocol=-1,
verbose_name="Result",
),
),
migrations.AlterField(
model_name="task",
name="started",
field=models.DateTimeField(
editable=False,
help_text="When the worker started executing the task.",
verbose_name="Started at",
),
),
migrations.AlterField(
model_name="task",
name="stopped",
field=models.DateTimeField(
editable=False,
help_text="When the worker finished (success or failure).",
verbose_name="Stopped at",
),
),
migrations.AlterField(
model_name="task",
name="success",
field=models.BooleanField(
default=True,
editable=False,
help_text="False if the task raised an exception or timed out.",
verbose_name="Success",
),
),
]

View File

@@ -1,36 +1,108 @@
from datetime import datetime, timedelta
from keyword import iskeyword
# Django
from django import get_version
from django.core.exceptions import ValidationError
from django.db import models
from django.template.defaultfilters import truncatechars
from django.db.models import Q
from django.urls import reverse
from django.utils import timezone
from django.utils.functional import cached_property
from django.utils.html import format_html
from django.utils.translation import gettext_lazy as _
# External
from picklefield import PickledObjectField
from picklefield.fields import dbsafe_decode
# Local
from django_q.conf import croniter
from django_q.signing import SignedPackage
from django_q.utils import add_months, add_years, localtime
from .utils import get_func_repr
class Task(models.Model):
id = models.CharField(max_length=32, primary_key=True, editable=False)
name = models.CharField(max_length=100, editable=False)
func = models.CharField(max_length=256)
hook = models.CharField(max_length=256, null=True)
args = PickledObjectField(null=True, protocol=-1)
kwargs = PickledObjectField(null=True, protocol=-1)
result = PickledObjectField(null=True, protocol=-1)
group = models.CharField(max_length=100, editable=False, null=True)
started = models.DateTimeField(editable=False)
stopped = models.DateTimeField(editable=False)
success = models.BooleanField(default=True, editable=False)
attempt_count = models.IntegerField(default=0)
id = models.CharField(
max_length=32,
primary_key=True,
editable=False,
verbose_name=_("Task id"),
help_text=_("32-character task identifier."),
)
name = models.CharField(
max_length=100,
editable=False,
verbose_name=_("Name"),
help_text=_("Optional human-readable name for lookup and display."),
)
func = models.CharField(
max_length=256,
verbose_name=_("Function"),
help_text=_("Dotted import path to the callable, e.g. myapp.tasks.job."),
)
hook = models.CharField(
max_length=256,
null=True,
verbose_name=_("Hook"),
help_text=_(
"Optional dotted path to a callback that receives the task result."
),
)
args = PickledObjectField(
null=True,
protocol=-1,
verbose_name=_("Arguments"),
help_text=_("Pickled positional arguments (tuple)."),
)
kwargs = PickledObjectField(
null=True,
protocol=-1,
verbose_name=_("Keyword arguments"),
help_text=_("Pickled keyword arguments (dict)."),
)
result = PickledObjectField(
null=True,
protocol=-1,
verbose_name=_("Result"),
help_text=_("Pickled return value or error payload."),
)
group = models.CharField(
max_length=100,
editable=False,
null=True,
verbose_name=_("Group"),
help_text=_("Optional group id to aggregate related tasks."),
)
cluster = models.CharField(
max_length=100,
default=None,
null=True,
blank=True,
verbose_name=_("Cluster"),
help_text=_("Name of the cluster that executed this task."),
)
started = models.DateTimeField(
editable=False,
verbose_name=_("Started at"),
help_text=_("When the worker started executing the task."),
)
stopped = models.DateTimeField(
editable=False,
verbose_name=_("Stopped at"),
help_text=_("When the worker finished (success or failure)."),
)
success = models.BooleanField(
default=True,
editable=False,
verbose_name=_("Success"),
help_text=_("False if the task raised an exception or timed out."),
)
attempt_count = models.IntegerField(
default=0,
verbose_name=_("Attempt count"),
help_text=_("How many times execution was attempted."),
)
@staticmethod
def get_result(task_id):
@@ -51,7 +123,7 @@ class Task(models.Model):
.exclude(success=False)
.values_list("result", flat=True)
)
return decode_results(values)
return values
def group_result(self, failures=False):
if self.group:
@@ -94,16 +166,21 @@ class Task(models.Model):
def time_taken(self):
return (self.stopped - self.started).total_seconds()
@property
def short_result(self):
return truncatechars(self.result, 100)
def __str__(self):
return f"{self.name or self.id}"
class Meta:
app_label = "django_q"
verbose_name = _("Task")
verbose_name_plural = _("Tasks")
ordering = ["-stopped"]
indexes = [
models.Index(
name="success_index",
fields=["group", "name", "func"],
condition=Q(success=True),
),
]
class SuccessManager(models.Manager):
@@ -148,18 +225,41 @@ def validate_cron(value):
raise ValidationError(e)
def validate_kwarg(value):
return value.isidentifier() and not iskeyword(value)
class Schedule(models.Model):
name = models.CharField(max_length=100, null=True, blank=True)
func = models.CharField(max_length=256, help_text="e.g. module.tasks.function")
name = models.CharField(
max_length=100,
null=True,
blank=True,
verbose_name=_("Name"),
help_text=_("Optional label to identify this schedule in the admin."),
)
func = models.CharField(
max_length=256,
verbose_name=_("Function"),
help_text=_("e.g. module.tasks.function"),
)
hook = models.CharField(
max_length=256,
null=True,
blank=True,
help_text="e.g. module.tasks.result_function",
verbose_name=_("Hook"),
help_text=_("e.g. module.tasks.result_function"),
)
args = models.TextField(
null=True,
blank=True,
verbose_name=_("Arguments"),
help_text=_("e.g. 1, 2, 'John'"),
)
args = models.TextField(null=True, blank=True, help_text=_("e.g. 1, 2, 'John'"))
kwargs = models.TextField(
null=True, blank=True, help_text=_("e.g. x=1, y=2, name='John'")
null=True,
blank=True,
verbose_name=_("Keyword arguments"),
help_text=_("e.g. x=1, y=2, name='John'"),
)
ONCE = "O"
MINUTES = "I"
@@ -186,26 +286,106 @@ class Schedule(models.Model):
(CRON, _("Cron")),
)
schedule_type = models.CharField(
max_length=2, choices=TYPE, default=TYPE[0][0], verbose_name=_("Schedule Type")
max_length=2,
choices=TYPE,
default=TYPE[0][0],
verbose_name=_("Schedule Type"),
help_text=_("How often this task should be enqueued."),
)
minutes = models.PositiveSmallIntegerField(
null=True, blank=True, help_text=_("Number of minutes for the Minutes type")
null=True,
blank=True,
verbose_name=_("Minutes"),
help_text=_("Number of minutes for the Minutes type"),
)
repeats = models.IntegerField(
default=-1, verbose_name=_("Repeats"), help_text=_("n = n times, -1 = forever")
)
next_run = models.DateTimeField(
verbose_name=_("Next Run"), default=timezone.now, null=True
verbose_name=_("Next Run"),
help_text=_("When this schedule runs next (stored in UTC)."),
default=timezone.now,
null=True,
)
cron = models.CharField(
max_length=100,
null=True,
blank=True,
validators=[validate_cron],
verbose_name=_("Cron"),
help_text=_("Cron expression"),
)
task = models.CharField(max_length=100, null=True, editable=False)
cluster = models.CharField(max_length=100, default=None, null=True, blank=True)
task = models.CharField(
max_length=100,
null=True,
editable=False,
verbose_name=_("Last task id"),
help_text=_("Id of the last task spawned from this schedule (read-only)."),
)
cluster = models.CharField(
max_length=100,
default=None,
null=True,
blank=True,
verbose_name=_("Cluster"),
help_text=_("Name of the target cluster"),
)
intended_date_kwarg = models.CharField(
max_length=100,
null=True,
blank=True,
validators=[validate_kwarg],
verbose_name=_("Intended date kwarg"),
help_text=_("Name of kwarg to pass intended schedule date"),
)
def calculate_next_run(self, next_run=None):
# next run is always in UTC
next_run = next_run or self.next_run
if self.schedule_type == self.CRON:
if not croniter:
raise ImportError(
_("Please install croniter to enable cron expressions")
)
return croniter(self.cron, localtime()).get_next(datetime)
if self.schedule_type == self.MINUTES:
add = timedelta(minutes=(self.minutes or 1))
elif self.schedule_type == self.HOURLY:
add = timedelta(hours=1)
elif self.schedule_type == self.DAILY:
add = timedelta(days=1)
elif self.schedule_type == self.WEEKLY:
add = timedelta(weeks=1)
elif self.schedule_type == self.BIWEEKLY:
add = timedelta(weeks=2)
elif self.schedule_type == self.MONTHLY:
add = timedelta(days=(add_months(next_run, 1) - next_run).days)
elif self.schedule_type == self.BIMONTHLY:
add = timedelta(days=(add_months(next_run, 2) - next_run).days)
elif self.schedule_type == self.QUARTERLY:
add = timedelta(days=(add_months(next_run, 3) - next_run).days)
elif self.schedule_type == self.YEARLY:
add = timedelta(days=(add_years(next_run, 1) - next_run).days)
# add normal timedelta, we will correct this later based on timezone
next_run += add
# DST differencers don't matter with minutes, hourly or yearly, so skip those
if self.schedule_type not in [self.MINUTES, self.HOURLY, self.YEARLY]:
# Get localtimes and then remove the tzinfo, so we can get the actual difference
current_next_run = localtime(next_run - add).replace(tzinfo=None)
new_next_run = localtime(next_run).replace(tzinfo=None)
# get the difference between them, this should be (-)1 or (-)0.5 hour
# based on DST active or not
extra_diff = (new_next_run - current_next_run) - add
# subtract difference
next_run -= extra_diff
return next_run
def success(self):
if self.task and Task.objects.filter(id=self.task):
@@ -218,18 +398,23 @@ class Schedule(models.Model):
url = reverse("admin:django_q_success_change", args=(task.id,))
else:
url = reverse("admin:django_q_failure_change", args=(task.id,))
return format_html(f'<a href="{url}">[{task.name}]</a>')
return format_html('<a href="{}">[{}]</a>', url, task.name)
return None
def __str__(self):
return self.func
def save(self, *args, **kwargs):
if self.pk is None and self.schedule_type == self.CRON:
self.next_run = self.calculate_next_run()
return super().save(*args, **kwargs)
success.boolean = True
success.short_description = _("success")
last_run.allow_tags = True
last_run.short_description = _("last_run")
class Meta:
app_label = "django_q"
verbose_name = _("Scheduled task")
@@ -238,34 +423,59 @@ class Schedule(models.Model):
class OrmQ(models.Model):
key = models.CharField(max_length=100)
payload = models.TextField()
lock = models.DateTimeField(null=True)
key = models.CharField(
max_length=100,
verbose_name=_("Cluster key"),
help_text=_("Name of the target cluster"),
)
payload = models.TextField(
verbose_name=_("Payload"),
help_text=_("Signed serialized task package (do not edit manually)."),
)
lock = models.DateTimeField(
null=True,
verbose_name=_("Lock until"),
help_text=_("Prevent any cluster from pulling until"),
)
@cached_property
def task(self):
return SignedPackage.loads(self.payload)
try:
return SignedPackage.loads(self.payload)
except Exception as e:
return {"id": "*" + e.__class__.__name__}
def func(self):
return get_func_repr(self.task()["func"])
return get_func_repr(self.task.get("func"))
def task_id(self):
return self.task()["id"]
return self.task.get("id")
def name(self):
return self.task()["name"]
return self.task.get("name")
def group(self):
return self.task().get("group")
return self.task.get("group")
def args(self):
return self.task.get("args")
def kwargs(self):
return self.task.get("kwargs")
def q_options(self):
exclude = {"id", "name", "group", "func", "args", "kwargs"}
return {k: v for k, v in self.task.items() if k not in exclude}
func.short_description = _("Function")
task_id.short_description = _("Task id")
name.short_description = _("Name")
group.short_description = _("Group")
args.short_description = _("Arguments")
kwargs.short_description = _("Keyword arguments")
q_options.short_description = _("Queue options")
class Meta:
app_label = "django_q"
verbose_name = _("Queued task")
verbose_name_plural = _("Queued tasks")
# Backwards compatibility for Django 1.7
def decode_results(values):
if get_version().split(".")[1] == "7":
# decode values in 1.7
return [dbsafe_decode(v) for v in values]
return values

View File

@@ -1,497 +1,212 @@
from datetime import timedelta
from multiprocessing.process import current_process
from multiprocessing.queues import Queue
# django
from django.db import connection
from django.db.models import F, Sum
from django.utils import timezone
from django.utils.translation import gettext as _
from django import core, db
from django.apps.registry import apps
from django.utils.translation import gettext_lazy as _
from django_q import VERSION, models
from django_q.brokers import get_broker
# local
from django_q.conf import Conf
from django_q.status import Stat
# optional
try:
import psutil
except ImportError:
psutil = None
apps.check_apps_ready()
except core.exceptions.AppRegistryNotReady:
import django
django.setup()
from django_q.brokers import Broker, get_broker
from django_q.conf import Conf, logger, setproctitle
from django_q.models import Success, Task
from django_q.signals import post_execute
from django_q.signing import SignedPackage
from django_q.tasks import async_chain
from django_q.utils import close_old_django_connections, get_func_repr
# optional
try:
from blessed import Terminal
except ImportError:
pass
import setproctitle
except ModuleNotFoundError:
setproctitle = None
def get_process_mb(pid):
try:
process = psutil.Process(pid)
mb_used = round(process.memory_info().rss / 1024 ** 2, 2)
except psutil.NoSuchProcess:
mb_used = "NO_PROCESS_FOUND"
return mb_used
BLESSED_INSTALL_MESSAGE = "Blessed is not installed. Please install blessed to use this: https://pypi.org/project/blessed/"
def monitor(run_once=False, broker=None):
def monitor(result_queue: Queue, broker: Broker = None):
"""
Gets finished tasks from the result queue and saves them to Django
:type broker: brokers.Broker
:type result_queue: multiprocessing.Queue
"""
if not broker:
broker = get_broker()
try:
term = Terminal()
except:
print(BLESSED_INSTALL_MESSAGE)
return
proc_name = current_process().name
if setproctitle:
setproctitle.setproctitle(f"qcluster {proc_name} monitor")
logger.info(
_("%(name)s monitoring at %(id)s")
% {"name": proc_name, "id": current_process().pid}
)
for task in iter(result_queue.get, "STOP"):
# save the result
if task.get("cached", False):
save_cached(task, broker)
else:
save_task(task, broker)
# acknowledge result
ack_id = task.pop("ack_id", False)
if ack_id and (task["success"] or task.get("ack_failure", False)):
broker.acknowledge(ack_id)
# signal execution done
post_execute.send(sender="django_q", task=task)
# log the result
info_name = get_func_repr(task["func"])
if task["success"]:
# log success
logger.info(
_("Processed '%(info_name)s' (%(task_name)s)")
% {"info_name": info_name, "task_name": task["name"]}
)
else:
# log failure
logger.error(
_("Failed '%(info_name)s' (%(task_name)s) - %(task_result)s")
% {
"info_name": info_name,
"task_name": task["name"],
"task_result": task["result"],
}
)
logger.info(_("%(name)s stopped monitoring results") % {"name": proc_name})
broker.ping()
with term.fullscreen(), term.hidden_cursor(), term.cbreak():
val = None
start_width = int(term.width / 8)
while val not in (
"q",
"Q",
def save_task(task, broker: Broker):
"""
Saves the task package to Django or the cache
:param task: the task package
:type broker: brokers.Broker
"""
# SAVE LIMIT < 0 : Don't save success
if not task.get("save", Conf.SAVE_LIMIT >= 0) and task["success"]:
return
# enqueues next in a chain
if task.get("chain", None):
async_chain(
task["chain"],
group=task["group"],
cached=task["cached"],
sync=task["sync"],
broker=broker,
)
# SAVE LIMIT > 0: Prune database, SAVE_LIMIT 0: No pruning
if not task.get("sync", False):
close_old_django_connections()
try:
filters = {}
if (
Conf.SAVE_LIMIT_PER
and Conf.SAVE_LIMIT_PER in {"group", "name", "func"}
and Conf.SAVE_LIMIT_PER in task
):
col_width = int(term.width / 8)
# In case of resize
if col_width != start_width:
print(term.clear())
start_width = col_width
print(
term.move(0, 0)
+ term.black_on_green(term.center(_("Host"), width=col_width - 1))
value = task[Conf.SAVE_LIMIT_PER]
if Conf.SAVE_LIMIT_PER == "func":
value = get_func_repr(value)
filters[Conf.SAVE_LIMIT_PER] = value
# check if we should clean the success tasks
if Conf.SAVE_LIMIT > 0:
with db.transaction.atomic(using=db.router.db_for_write(Success)):
success_tasks_qs = Success.objects.filter(**filters)
success_tasks_pks = [
success_task.pk
for success_task in success_tasks_qs.select_for_update()
]
if task["success"] and len(success_tasks_pks) >= Conf.SAVE_LIMIT:
success_tasks_qs.last().delete()
# check if this task has previous results
try:
task_obj = Task.objects.get(id=task["id"], name=task["name"])
# only update the result if it hasn't succeeded yet
if not task_obj.success:
task_obj.stopped = task["stopped"]
task_obj.result = task["result"]
task_obj.success = task["success"]
task_obj.attempt_count = task_obj.attempt_count + 1
task_obj.save()
except Task.DoesNotExist:
# convert func to string
func = get_func_repr(task["func"])
task_obj = Task.objects.create(
id=task["id"],
name=task["name"],
func=func,
hook=task.get("hook"),
args=task["args"],
kwargs=task["kwargs"],
cluster=task.get("cluster"),
started=task["started"],
stopped=task["stopped"],
result=task["result"],
group=task.get("group"),
success=task["success"],
attempt_count=1,
)
print(
term.move(0, 1 * col_width)
+ term.black_on_green(term.center(_("Id"), width=col_width - 1))
)
print(
term.move(0, 2 * col_width)
+ term.black_on_green(term.center(_("State"), width=col_width - 1))
)
print(
term.move(0, 3 * col_width)
+ term.black_on_green(term.center(_("Pool"), width=col_width - 1))
)
print(
term.move(0, 4 * col_width)
+ term.black_on_green(term.center(_("TQ"), width=col_width - 1))
)
print(
term.move(0, 5 * col_width)
+ term.black_on_green(term.center(_("RQ"), width=col_width - 1))
)
print(
term.move(0, 6 * col_width)
+ term.black_on_green(term.center(_("RC"), width=col_width - 1))
)
print(
term.move(0, 7 * col_width)
+ term.black_on_green(term.center(_("Up"), width=col_width - 1))
)
i = 2
stats = Stat.get_all(broker=broker)
print(term.clear_eos())
for stat in stats:
status = stat.status
# color status
if stat.status == Conf.WORKING:
status = term.green(str(Conf.WORKING))
elif stat.status == Conf.STOPPING:
status = term.yellow(str(Conf.STOPPING))
elif stat.status == Conf.STOPPED:
status = term.red(str(Conf.STOPPED))
elif stat.status == Conf.IDLE:
status = str(Conf.IDLE)
# color q's
tasks = str(stat.task_q_size)
if stat.task_q_size > 0:
tasks = term.cyan(str(stat.task_q_size))
if Conf.QUEUE_LIMIT and stat.task_q_size == Conf.QUEUE_LIMIT:
tasks = term.green(str(stat.task_q_size))
results = stat.done_q_size
if results > 0:
results = term.cyan(str(results))
# color workers
workers = len(stat.workers)
if workers < Conf.WORKERS:
workers = term.yellow(str(workers))
# format uptime
uptime = (timezone.now() - stat.tob).total_seconds()
hours, remainder = divmod(uptime, 3600)
minutes, seconds = divmod(remainder, 60)
uptime = "%d:%02d:%02d" % (hours, minutes, seconds)
# print to the terminal
print(
term.move(i, 0)
+ term.center(stat.host[: col_width - 1], width=col_width - 1)
)
print(
term.move(i, 1 * col_width)
+ term.center(str(stat.cluster_id)[-8:], width=col_width - 1)
)
print(
term.move(i, 2 * col_width)
+ term.center(status, width=col_width - 1)
)
print(
term.move(i, 3 * col_width)
+ term.center(workers, width=col_width - 1)
)
print(
term.move(i, 4 * col_width)
+ term.center(tasks, width=col_width - 1)
)
print(
term.move(i, 5 * col_width)
+ term.center(results, width=col_width - 1)
)
print(
term.move(i, 6 * col_width)
+ term.center(stat.reincarnations, width=col_width - 1)
)
print(
term.move(i, 7 * col_width)
+ term.center(uptime, width=col_width - 1)
)
i += 1
# bottom bar
i += 1
queue_size = broker.queue_size()
lock_size = broker.lock_size()
if lock_size:
queue_size = f"{queue_size}({lock_size})"
print(
term.move(i, 0)
+ term.white_on_cyan(term.center(broker.info(), width=col_width * 2))
)
print(
term.move(i, 2 * col_width)
+ term.black_on_cyan(term.center(_("Queued"), width=col_width))
)
print(
term.move(i, 3 * col_width)
+ term.white_on_cyan(term.center(queue_size, width=col_width))
)
print(
term.move(i, 4 * col_width)
+ term.black_on_cyan(term.center(_("Success"), width=col_width))
)
print(
term.move(i, 5 * col_width)
+ term.white_on_cyan(
term.center(models.Success.objects.count(), width=col_width)
)
)
print(
term.move(i, 6 * col_width)
+ term.black_on_cyan(term.center(_("Failures"), width=col_width))
)
print(
term.move(i, 7 * col_width)
+ term.white_on_cyan(
term.center(models.Failure.objects.count(), width=col_width)
)
)
# for testing
if run_once:
return Stat.get_all(broker=broker)
print(term.move(i + 2, 0) + term.center(_("[Press q to quit]")))
val = term.inkey(timeout=1)
if (
Conf.MAX_ATTEMPTS > 0
and task_obj.attempt_count >= Conf.MAX_ATTEMPTS
and task.get("ack_id")
):
broker.acknowledge(task["ack_id"])
except Exception:
logger.exception("Could not save task result")
def info(broker=None):
if not broker:
broker = get_broker()
def save_cached(task, broker: Broker):
task_key = f"{broker.list_key}:{task['id']}"
timeout = task["cached"]
if timeout is True:
timeout = None
try:
term = Terminal()
except:
print(BLESSED_INSTALL_MESSAGE)
return
broker.ping()
stat = Stat.get_all(broker=broker)
# general stats
clusters = len(stat)
workers = 0
reincarnations = 0
for cluster in stat:
workers += len(cluster.workers)
reincarnations += cluster.reincarnations
# calculate tasks pm and avg exec time
tasks_per = 0
per = _("day")
exec_time = 0
last_tasks = models.Success.objects.filter(
stopped__gte=timezone.now() - timedelta(hours=24)
)
tasks_per_day = last_tasks.count()
if tasks_per_day > 0:
# average execution time over the last 24 hours
if connection.vendor != "sqlite":
exec_time = last_tasks.aggregate(
time_taken=Sum(F("stopped") - F("started"))
)
exec_time = exec_time["time_taken"].total_seconds() / tasks_per_day
else:
# can't sum timedeltas on sqlite
for t in last_tasks:
exec_time += t.time_taken()
exec_time = exec_time / tasks_per_day
# tasks per second/minute/hour/day in the last 24 hours
if tasks_per_day > 24 * 60 * 60:
tasks_per = tasks_per_day / (24 * 60 * 60)
per = _("second")
elif tasks_per_day > 24 * 60:
tasks_per = tasks_per_day / (24 * 60)
per = _("minute")
elif tasks_per_day > 24:
tasks_per = tasks_per_day / 24
per = _("hour")
else:
tasks_per = tasks_per_day
# print to terminal
print(term.clear_eos())
col_width = int(term.width / 6)
print(
term.black_on_green(
term.center(
_(
'-- %(prefix)s %(version)s on %(info)s --'
) % {'prefix': Conf.PREFIX.capitalize(), 'version': ".".join(str(v) for v in VERSION), 'info': broker.info()}
)
)
)
print(
term.cyan(_("Clusters"))
+ term.move_x(1 * col_width)
+ term.white(str(clusters))
+ term.move_x(2 * col_width)
+ term.cyan(_("Workers"))
+ term.move_x(3 * col_width)
+ term.white(str(workers))
+ term.move_x(4 * col_width)
+ term.cyan(_("Restarts"))
+ term.move_x(5 * col_width)
+ term.white(str(reincarnations))
)
print(
term.cyan(_("Queued"))
+ term.move_x(1 * col_width)
+ term.white(str(broker.queue_size()))
+ term.move_x(2 * col_width)
+ term.cyan(_("Successes"))
+ term.move_x(3 * col_width)
+ term.white(str(models.Success.objects.count()))
+ term.move_x(4 * col_width)
+ term.cyan(_("Failures"))
+ term.move_x(5 * col_width)
+ term.white(str(models.Failure.objects.count()))
)
print(
term.cyan(_("Schedules"))
+ term.move_x(1 * col_width)
+ term.white(str(models.Schedule.objects.count()))
+ term.move_x(2 * col_width)
+ term.cyan(_("Tasks/%(per)s") % {'per': per})
+ term.move_x(3 * col_width)
+ term.white(f"{tasks_per:.2f}")
+ term.move_x(4 * col_width)
+ term.cyan(_("Avg time"))
+ term.move_x(5 * col_width)
+ term.white(f"{exec_time:.4f}")
)
return True
def memory(run_once=False, workers=False, broker=None):
if not broker:
broker = get_broker()
try:
term = Terminal()
except:
print(BLESSED_INSTALL_MESSAGE)
return
broker.ping()
if not psutil:
print(term.clear_eos())
print(
term.white_on_red(
'Cannot start "qmemory" command. Missing "psutil" library.'
)
)
return
with term.fullscreen(), term.hidden_cursor(), term.cbreak():
MEMORY_AVAILABLE_LOWEST_PERCENTAGE = 100.0
MEMORY_AVAILABLE_LOWEST_PERCENTAGE_AT = timezone.now()
cols = 8
val = None
start_width = int(term.width / cols)
while val not in ["q", "Q"]:
col_width = int(term.width / cols)
# In case of resize
if col_width != start_width:
print(term.clear())
start_width = col_width
# sentinel, monitor and workers memory usage
print(
term.move(0, 0 * col_width)
+ term.black_on_green(term.center(_("Host"), width=col_width - 1))
)
print(
term.move(0, 1 * col_width)
+ term.black_on_green(term.center(_("Id"), width=col_width - 1))
)
print(
term.move(0, 2 * col_width)
+ term.black_on_green(
term.center(_("Available (%)"), width=col_width - 1)
group = task.get("group", None)
iter_count = task.get("iter_count", 0)
# if it's a group append to the group list
if group:
group_key = f"{broker.list_key}:{group}:keys"
group_list = broker.cache.get(group_key) or []
# if it's an iter group, check if we are ready
if iter_count and len(group_list) == iter_count - 1:
group_args = f"{broker.list_key}:{group}:args"
# collate the results into a Task result
results = [
SignedPackage.loads(broker.cache.get(k))["result"]
for k in group_list
]
results.append(task["result"])
task["result"] = results
task["id"] = group
task["args"] = SignedPackage.loads(broker.cache.get(group_args))
task.pop("iter_count", None)
task.pop("group", None)
if task.get("iter_cached", None):
task["cached"] = task.pop("iter_cached", None)
save_cached(task, broker=broker)
else:
save_task(task, broker)
broker.cache.delete_many(group_list)
broker.cache.delete_many([group_key, group_args])
return
# save the group list
group_list.append(task_key)
broker.cache.set(group_key, group_list, timeout)
# async_task next in a chain
if task.get("chain", None):
async_chain(
task["chain"],
group=group,
cached=task["cached"],
sync=task["sync"],
broker=broker,
)
)
print(
term.move(0, 3 * col_width)
+ term.black_on_green(
term.center(_("Available (MB)"), width=col_width - 1)
)
)
print(
term.move(0, 4 * col_width)
+ term.black_on_green(term.center(_("Total (MB)"), width=col_width - 1))
)
print(
term.move(0, 5 * col_width)
+ term.black_on_green(
term.center(_("Sentinel (MB)"), width=col_width - 1)
)
)
print(
term.move(0, 6 * col_width)
+ term.black_on_green(
term.center(_("Monitor (MB)"), width=col_width - 1)
)
)
print(
term.move(0, 7 * col_width)
+ term.black_on_green(
term.center(_("Workers (MB)"), width=col_width - 1)
)
)
row = 2
stats = Stat.get_all(broker=broker)
print(term.clear_eos())
for stat in stats:
# memory available (%)
memory_available_percentage = round(
psutil.virtual_memory().available
* 100
/ psutil.virtual_memory().total,
2,
)
# memory available (MB)
memory_available = round(
psutil.virtual_memory().available / 1024 ** 2, 2
)
if memory_available_percentage < MEMORY_AVAILABLE_LOWEST_PERCENTAGE:
MEMORY_AVAILABLE_LOWEST_PERCENTAGE = memory_available_percentage
MEMORY_AVAILABLE_LOWEST_PERCENTAGE_AT = timezone.now()
print(
term.move(row, 0 * col_width)
+ term.center(stat.host[: col_width - 1], width=col_width - 1)
)
print(
term.move(row, 1 * col_width)
+ term.center(str(stat.cluster_id)[-8:], width=col_width - 1)
)
print(
term.move(row, 2 * col_width)
+ term.center(memory_available_percentage, width=col_width - 1)
)
print(
term.move(row, 3 * col_width)
+ term.center(memory_available, width=col_width - 1)
)
print(
term.move(row, 4 * col_width)
+ term.center(
round(psutil.virtual_memory().total / 1024 ** 2, 2),
width=col_width - 1,
)
)
print(
term.move(row, 5 * col_width)
+ term.center(get_process_mb(stat.sentinel), width=col_width - 1)
)
print(
term.move(row, 6 * col_width)
+ term.center(
get_process_mb(getattr(stat, "monitor", None)),
width=col_width - 1,
)
)
workers_mb = 0
for worker_pid in stat.workers:
result = get_process_mb(worker_pid)
if isinstance(result, str):
result = 0
workers_mb += result
print(
term.move(row, 7 * col_width)
+ term.center(
workers_mb or "NO_PROCESSES_FOUND", width=col_width - 1
)
)
row += 1
# each worker's memory usage
if workers:
row += 2
col_width = int(term.width / (1 + Conf.WORKERS))
print(
term.move(row, 0 * col_width)
+ term.black_on_cyan(term.center(_("Id"), width=col_width - 1))
)
for worker_num in range(Conf.WORKERS):
print(
term.move(row, (worker_num + 1) * col_width)
+ term.black_on_cyan(
term.center(
"Worker #{} (MB)".format(worker_num + 1),
width=col_width - 1,
)
)
)
row += 2
for stat in stats:
print(
term.move(row, 0 * col_width)
+ term.center(str(stat.cluster_id)[-8:], width=col_width - 1)
)
for idx, worker_pid in enumerate(stat.workers):
mb_used = get_process_mb(worker_pid)
print(
term.move(row, (idx + 1) * col_width)
+ term.center(mb_used, width=col_width - 1)
)
row += 1
row += 1
print(
term.move(row, 0)
+ _("Available lowest (): %(memory_percent)s ((at)s)") % { 'memory_percent': str(MEMORY_AVAILABLE_LOWEST_PERCENTAGE), 'at': MEMORY_AVAILABLE_LOWEST_PERCENTAGE_AT.strftime(
"%Y-%m-%d %H:%M:%S+00:00"
)}
)
# for testing
if run_once:
return Stat.get_all(broker=broker)
print(term.move(row + 2, 0) + term.center(_("[Press q to quit]")))
val = term.inkey(timeout=1)
def get_ids():
# prints id (PID) of running clusters
stat = Stat.get_all()
if stat:
for s in stat:
print(s.cluster_id)
else:
print(_("No clusters appear to be running."))
return True
# save the task
broker.cache.set(task_key, SignedPackage.dumps(task), timeout)
except Exception:
logger.exception("Could not save task result")

View File

@@ -0,0 +1,508 @@
from datetime import timedelta
# django
from django.db import connection
from django.db.models import F, Sum
from django.utils import timezone
from django.utils.translation import gettext as _
from django_q import VERSION, models
from django_q.brokers import get_broker
# local
from django_q.conf import Conf
from django_q.status import Stat
# optional
try:
import psutil
except ImportError:
psutil = None
def get_process_mb(pid):
try:
process = psutil.Process(pid)
mb_used = round(process.memory_info().rss / 1024**2, 2)
except psutil.NoSuchProcess:
mb_used = "NO_PROCESS_FOUND"
return mb_used
BLESSED_INSTALL_MESSAGE = (
"Blessed is not installed. Please install blessed to use this: "
"https://pypi.org/project/blessed/"
)
def monitor(run_once=False, broker=None):
if not broker:
broker = get_broker()
try:
from blessed import Terminal
term = Terminal()
except ImportError:
print(BLESSED_INSTALL_MESSAGE)
return
broker.ping()
with term.fullscreen(), term.hidden_cursor(), term.cbreak():
val = None
start_width = int(term.width / 8)
while val not in (
"q",
"Q",
):
col_width = int(term.width / 8)
# In case of resize
if col_width != start_width:
print(term.clear())
start_width = col_width
print(
term.move(0, 0)
+ term.black_on_green(term.center(_("Host"), width=col_width - 1))
)
print(
term.move(0, 1 * col_width)
+ term.black_on_green(term.center(_("Id"), width=col_width - 1))
)
print(
term.move(0, 2 * col_width)
+ term.black_on_green(term.center(_("State"), width=col_width - 1))
)
print(
term.move(0, 3 * col_width)
+ term.black_on_green(term.center(_("Pool"), width=col_width - 1))
)
print(
term.move(0, 4 * col_width)
+ term.black_on_green(term.center(_("TQ"), width=col_width - 1))
)
print(
term.move(0, 5 * col_width)
+ term.black_on_green(term.center(_("RQ"), width=col_width - 1))
)
print(
term.move(0, 6 * col_width)
+ term.black_on_green(term.center(_("RC"), width=col_width - 1))
)
print(
term.move(0, 7 * col_width)
+ term.black_on_green(term.center(_("Up"), width=col_width - 1))
)
i = 2
stats = Stat.get_all(broker=broker)
print(term.clear_eos())
for stat in stats:
status = stat.status
# color status
if stat.status == Conf.WORKING:
status = term.green(str(Conf.WORKING))
elif stat.status == Conf.STOPPING:
status = term.yellow(str(Conf.STOPPING))
elif stat.status == Conf.STOPPED:
status = term.red(str(Conf.STOPPED))
elif stat.status == Conf.IDLE:
status = str(Conf.IDLE)
# color q's
tasks = str(stat.task_q_size)
if stat.task_q_size > 0:
tasks = term.cyan(str(stat.task_q_size))
if Conf.QUEUE_LIMIT and stat.task_q_size == Conf.QUEUE_LIMIT:
tasks = term.green(str(stat.task_q_size))
results = stat.done_q_size
if results > 0:
results = term.cyan(str(results))
# color workers
workers = len(stat.workers)
if workers < Conf.WORKERS:
workers = term.yellow(str(workers))
# format uptime
uptime = (timezone.now() - stat.tob).total_seconds()
hours, remainder = divmod(uptime, 3600)
minutes, seconds = divmod(remainder, 60)
uptime = "%d:%02d:%02d" % (hours, minutes, seconds)
# print to the terminal
print(
term.move(i, 0)
+ term.center(stat.host[: col_width - 1], width=col_width - 1)
)
print(
term.move(i, 1 * col_width)
+ term.center(str(stat.cluster_id)[-8:], width=col_width - 1)
)
print(
term.move(i, 2 * col_width)
+ term.center(status, width=col_width - 1)
)
print(
term.move(i, 3 * col_width)
+ term.center(workers, width=col_width - 1)
)
print(
term.move(i, 4 * col_width)
+ term.center(tasks, width=col_width - 1)
)
print(
term.move(i, 5 * col_width)
+ term.center(results, width=col_width - 1)
)
print(
term.move(i, 6 * col_width)
+ term.center(stat.reincarnations, width=col_width - 1)
)
print(
term.move(i, 7 * col_width)
+ term.center(uptime, width=col_width - 1)
)
i += 1
# bottom bar
i += 1
queue_size = broker.queue_size()
lock_size = broker.lock_size()
if lock_size:
queue_size = f"{queue_size}({lock_size})"
print(
term.move(i, 0)
+ term.white_on_cyan(term.center(broker.info(), width=col_width * 2))
)
print(
term.move(i, 2 * col_width)
+ term.black_on_cyan(term.center(_("Queued"), width=col_width))
)
print(
term.move(i, 3 * col_width)
+ term.white_on_cyan(term.center(str(queue_size), width=col_width))
)
print(
term.move(i, 4 * col_width)
+ term.black_on_cyan(term.center(_("Success"), width=col_width))
)
print(
term.move(i, 5 * col_width)
+ term.white_on_cyan(
term.center(str(models.Success.objects.count()), width=col_width)
)
)
print(
term.move(i, 6 * col_width)
+ term.black_on_cyan(term.center(_("Failures"), width=col_width))
)
print(
term.move(i, 7 * col_width)
+ term.white_on_cyan(
term.center(str(models.Failure.objects.count()), width=col_width)
)
)
# for testing
if run_once:
return Stat.get_all(broker=broker)
print(term.move(i + 2, 0) + term.center(_("[Press q to quit]")))
val = term.inkey(timeout=1)
def info(broker=None):
if not broker:
broker = get_broker()
try:
from blessed import Terminal
term = Terminal()
except ImportError:
print(BLESSED_INSTALL_MESSAGE)
return
broker.ping()
stat = Stat.get_all(broker=broker)
# general stats
clusters = len(stat)
workers = 0
reincarnations = 0
for cluster in stat:
workers += len(cluster.workers)
reincarnations += cluster.reincarnations
# calculate tasks pm and avg exec time
tasks_per = 0
per = _("day")
exec_time = 0
last_tasks = models.Success.objects.filter(
stopped__gte=timezone.now() - timedelta(hours=24)
)
tasks_per_day = last_tasks.count()
if tasks_per_day > 0:
# average execution time over the last 24 hours
if connection.vendor != "sqlite":
exec_time = last_tasks.aggregate(
time_taken=Sum(F("stopped") - F("started"))
)
exec_time = exec_time["time_taken"].total_seconds() / tasks_per_day
else:
# can't sum timedeltas on sqlite
for t in last_tasks:
exec_time += t.time_taken()
exec_time = exec_time / tasks_per_day
# tasks per second/minute/hour/day in the last 24 hours
if tasks_per_day > 24 * 60 * 60:
tasks_per = tasks_per_day / (24 * 60 * 60)
per = _("second")
elif tasks_per_day > 24 * 60:
tasks_per = tasks_per_day / (24 * 60)
per = _("minute")
elif tasks_per_day > 24:
tasks_per = tasks_per_day / 24
per = _("hour")
else:
tasks_per = tasks_per_day
# print to terminal
print(term.clear_eos())
col_width = int(term.width / 6)
print(
term.black_on_green(
term.center(
_("-- %(prefix)s %(version)s on %(info)s --")
% {
"prefix": Conf.PREFIX.capitalize(),
"version": ".".join(str(v) for v in VERSION),
"info": broker.info(),
}
)
)
)
print(
term.cyan(_("Clusters"))
+ term.move_x(1 * col_width)
+ term.white(str(clusters))
+ term.move_x(2 * col_width)
+ term.cyan(_("Workers"))
+ term.move_x(3 * col_width)
+ term.white(str(workers))
+ term.move_x(4 * col_width)
+ term.cyan(_("Restarts"))
+ term.move_x(5 * col_width)
+ term.white(str(reincarnations))
)
print(
term.cyan(_("Queued"))
+ term.move_x(1 * col_width)
+ term.white(str(broker.queue_size()))
+ term.move_x(2 * col_width)
+ term.cyan(_("Successes"))
+ term.move_x(3 * col_width)
+ term.white(str(models.Success.objects.count()))
+ term.move_x(4 * col_width)
+ term.cyan(_("Failures"))
+ term.move_x(5 * col_width)
+ term.white(str(models.Failure.objects.count()))
)
print(
term.cyan(_("Schedules"))
+ term.move_x(1 * col_width)
+ term.white(str(models.Schedule.objects.count()))
+ term.move_x(2 * col_width)
+ term.cyan(_("Tasks/%(per)s") % {"per": per})
+ term.move_x(3 * col_width)
+ term.white(f"{tasks_per:.2f}")
+ term.move_x(4 * col_width)
+ term.cyan(_("Avg time"))
+ term.move_x(5 * col_width)
+ term.white(f"{exec_time:.4f}")
)
return True
def memory(run_once=False, workers=False, broker=None):
if not broker:
broker = get_broker()
try:
from blessed import Terminal
term = Terminal()
except ImportError:
print(BLESSED_INSTALL_MESSAGE)
return
broker.ping()
if not psutil:
print(term.clear_eos())
print(
term.white_on_red(
'Cannot start "qmemory" command. Missing "psutil" library.'
)
)
return
with term.fullscreen(), term.hidden_cursor(), term.cbreak():
MEMORY_AVAILABLE_LOWEST_PERCENTAGE = 100.0
MEMORY_AVAILABLE_LOWEST_PERCENTAGE_AT = timezone.now()
cols = 8
val = None
start_width = int(term.width / cols)
while val not in ["q", "Q"]:
col_width = int(term.width / cols)
# In case of resize
if col_width != start_width:
print(term.clear())
start_width = col_width
# sentinel, monitor and workers memory usage
print(
term.move(0, 0 * col_width)
+ term.black_on_green(term.center(_("Host"), width=col_width - 1))
)
print(
term.move(0, 1 * col_width)
+ term.black_on_green(term.center(_("Id"), width=col_width - 1))
)
print(
term.move(0, 2 * col_width)
+ term.black_on_green(
term.center(_("Available (%)"), width=col_width - 1)
)
)
print(
term.move(0, 3 * col_width)
+ term.black_on_green(
term.center(_("Available (MB)"), width=col_width - 1)
)
)
print(
term.move(0, 4 * col_width)
+ term.black_on_green(term.center(_("Total (MB)"), width=col_width - 1))
)
print(
term.move(0, 5 * col_width)
+ term.black_on_green(
term.center(_("Sentinel (MB)"), width=col_width - 1)
)
)
print(
term.move(0, 6 * col_width)
+ term.black_on_green(
term.center(_("Monitor (MB)"), width=col_width - 1)
)
)
print(
term.move(0, 7 * col_width)
+ term.black_on_green(
term.center(_("Workers (MB)"), width=col_width - 1)
)
)
row = 2
stats = Stat.get_all(broker=broker)
print(term.clear_eos())
for stat in stats:
# memory available (%)
memory_available_percentage = round(
psutil.virtual_memory().available
* 100
/ psutil.virtual_memory().total,
2,
)
# memory available (MB)
memory_available = round(psutil.virtual_memory().available / 1024**2, 2)
if memory_available_percentage < MEMORY_AVAILABLE_LOWEST_PERCENTAGE:
MEMORY_AVAILABLE_LOWEST_PERCENTAGE = memory_available_percentage
MEMORY_AVAILABLE_LOWEST_PERCENTAGE_AT = timezone.now()
print(
term.move(row, 0 * col_width)
+ term.center(stat.host[: col_width - 1], width=col_width - 1)
)
print(
term.move(row, 1 * col_width)
+ term.center(str(stat.cluster_id)[-8:], width=col_width - 1)
)
print(
term.move(row, 2 * col_width)
+ term.center(memory_available_percentage, width=col_width - 1)
)
print(
term.move(row, 3 * col_width)
+ term.center(memory_available, width=col_width - 1)
)
print(
term.move(row, 4 * col_width)
+ term.center(
round(psutil.virtual_memory().total / 1024**2, 2),
width=col_width - 1,
)
)
print(
term.move(row, 5 * col_width)
+ term.center(get_process_mb(stat.sentinel), width=col_width - 1)
)
print(
term.move(row, 6 * col_width)
+ term.center(
get_process_mb(getattr(stat, "monitor", None)),
width=col_width - 1,
)
)
workers_mb = 0
for worker_pid in stat.workers:
result = get_process_mb(worker_pid)
if isinstance(result, str):
result = 0
workers_mb += result
print(
term.move(row, 7 * col_width)
+ term.center(
workers_mb or "NO_PROCESSES_FOUND", width=col_width - 1
)
)
row += 1
# each worker's memory usage
if workers:
row += 2
col_width = int(term.width / (1 + Conf.WORKERS))
print(
term.move(row, 0 * col_width)
+ term.black_on_cyan(term.center(_("Id"), width=col_width - 1))
)
for worker_num in range(Conf.WORKERS):
print(
term.move(row, (worker_num + 1) * col_width)
+ term.black_on_cyan(
term.center(
"Worker #{} (MB)".format(worker_num + 1),
width=col_width - 1,
)
)
)
row += 2
for stat in stats:
print(
term.move(row, 0 * col_width)
+ term.center(str(stat.cluster_id)[-8:], width=col_width - 1)
)
for idx, worker_pid in enumerate(stat.workers):
mb_used = get_process_mb(worker_pid)
print(
term.move(row, (idx + 1) * col_width)
+ term.center(mb_used, width=col_width - 1)
)
row += 1
row += 1
print(
term.move(row, 0)
+ _("Available lowest (): %(memory_percent)s ((at)s)")
% {
"memory_percent": str(MEMORY_AVAILABLE_LOWEST_PERCENTAGE),
"at": MEMORY_AVAILABLE_LOWEST_PERCENTAGE_AT.strftime(
"%Y-%m-%d %H:%M:%S+00:00"
),
}
)
# for testing
if run_once:
return Stat.get_all(broker=broker)
print(term.move(row + 2, 0) + term.center(_("[Press q to quit]")))
val = term.inkey(timeout=1)
def get_ids():
# prints id (PID) of running clusters
stat = Stat.get_all()
if stat:
for s in stat:
print(s.cluster_id)
else:
print(_("No clusters appear to be running."))
return True

71
django_q/pusher.py Normal file
View File

@@ -0,0 +1,71 @@
from multiprocessing import Event
from multiprocessing.process import current_process
from multiprocessing.queues import Queue
from time import sleep
from django import core
from django.apps.registry import apps
from django.utils.translation import gettext_lazy as _
try:
apps.check_apps_ready()
except core.exceptions.AppRegistryNotReady:
import django
django.setup()
from django_q.brokers import Broker, get_broker
from django_q.conf import Conf, logger
from django_q.signing import BadSignature, SignedPackage
try:
import setproctitle
except ModuleNotFoundError:
setproctitle = None
def pusher(task_queue: Queue, event: Event, broker: Broker = None):
"""
Pulls tasks of the broker and puts them in the task queue
:type broker:
:type task_queue: multiprocessing.Queue
:type event: multiprocessing.Event
"""
if not broker:
broker = get_broker()
proc_name = current_process().name
if setproctitle:
setproctitle.setproctitle(f"qcluster {proc_name} pusher")
logger.info(
_("%(name)s pushing tasks at %(id)s")
% {"name": proc_name, "id": current_process().pid}
)
while True:
try:
task_set = broker.dequeue()
except Exception:
logger.exception("Failed to pull task from broker")
# broker probably crashed. Let the sentinel handle it.
sleep(10)
break
if task_set:
for task in task_set:
ack_id = task[0]
# unpack the task
try:
task = SignedPackage.loads(task[1])
except (TypeError, BadSignature):
logger.exception("Failed to push task to queue")
broker.fail(ack_id)
continue
task["cluster"] = (
Conf.CLUSTER_NAME
) # save actual cluster name to orm task table
task["ack_id"] = ack_id
task_queue.put(task)
logger.debug(
_("queueing from %(list_key)s") % {"list_key": broker.list_key}
)
if event.is_set():
break
logger.info(_("%(name)s stopped pushing tasks") % {"name": current_process().name})

View File

@@ -1,6 +1,8 @@
"""
The code is derived from https://github.com/althonos/pronto/commit/3384010dfb4fc7c66a219f59276adef3288a886b
The code is derived from
https://github.com/althonos/pronto/commit/3384010dfb4fc7c66a219f59276adef3288a886b
"""
import multiprocessing
import multiprocessing.queues
import sys

140
django_q/scheduler.py Normal file
View File

@@ -0,0 +1,140 @@
import ast
from multiprocessing.process import current_process
from django import core, db
from django.apps.registry import apps
from django.utils import timezone
from django.utils.translation import gettext_lazy as _
try:
apps.check_apps_ready()
except core.exceptions.AppRegistryNotReady:
import django
django.setup()
from django_q.brokers import Broker, get_broker
from django_q.conf import Conf, logger
from django_q.humanhash import humanize
from django_q.models import Schedule
from django_q.tasks import async_task
from django_q.utils import close_old_django_connections, localtime
def scheduler(broker: Broker = None):
"""
Creates a task from a schedule at the scheduled time and schedules next run
"""
if not broker:
broker = get_broker()
close_old_django_connections()
try:
# Only default cluster will handler schedule with default(null) cluster
Q_default = (
db.models.Q(cluster__isnull=True)
if Conf.CLUSTER_NAME == Conf.PREFIX
else db.models.Q(pk__in=[])
)
with db.transaction.atomic(using=db.router.db_for_write(Schedule)):
for s in (
Schedule.objects.select_for_update()
.exclude(repeats=0)
.filter(next_run__lt=timezone.now())
.filter(Q_default | db.models.Q(cluster=Conf.CLUSTER_NAME))
):
args = ()
kwargs = {}
# get args, kwargs and hook
if s.kwargs:
try:
# first try the dict syntax
kwargs = ast.literal_eval(s.kwargs)
except (SyntaxError, ValueError):
# else use the kwargs syntax
try:
parsed_kwargs = (
ast.parse(f"f({s.kwargs})").body[0].value.keywords
)
kwargs = {
kwarg.arg: ast.literal_eval(kwarg.value)
for kwarg in parsed_kwargs
}
except (SyntaxError, ValueError):
kwargs = {}
if s.args:
args = ast.literal_eval(s.args)
# single value won't eval to tuple, so:
if type(args) is not tuple:
args = (args,)
q_options = kwargs.get("q_options", {})
if s.intended_date_kwarg:
kwargs[s.intended_date_kwarg] = s.next_run.isoformat()
if s.hook:
q_options["hook"] = s.hook
# set up the next run time
if s.schedule_type != s.ONCE:
next_run = s.next_run
while True:
next_run = s.calculate_next_run(next_run)
if Conf.CATCH_UP or next_run > localtime():
break
s.next_run = next_run
# Little Fix for already broken numbers
if s.repeats < -1:
s.repeats = -1
# Check if the value is not zero
if s.repeats > 0:
s.repeats -= 1
# send it to the cluster; any cluster name is allowed in multi-queue scenarios
# because `broker_name` is confusing, using `cluster` name is recommended and takes precedence
q_options["cluster"] = s.cluster or q_options.get(
"cluster", q_options.pop("broker_name", None)
)
if (
q_options["cluster"] is None
or q_options["cluster"] == Conf.CLUSTER_NAME
):
q_options["broker"] = broker
q_options["group"] = q_options.get("group", s.name or s.id)
kwargs["q_options"] = q_options
s.task = async_task(s.func, *args, **kwargs)
# log it
if not s.task:
logger.error(
_(
"%(process_name)s failed to create a task from schedule "
"[%(schedule)s]"
)
% {
"process_name": current_process().name,
"schedule": s.name or s.id,
}
)
else:
logger.info(
_(
"%(process_name)s created task %(task_name)s from schedule "
"[%(schedule)s]"
)
% {
"process_name": current_process().name,
"task_name": humanize(s.task),
"schedule": s.name or s.id,
}
)
# default behavior is to delete a ONCE schedule
if s.schedule_type == s.ONCE:
if s.repeats < 0:
s.delete()
continue
# but not if it has a positive repeats
s.repeats = 0
# save the schedule
s.save()
except Exception:
logger.exception("Could not create task from schedule")

View File

@@ -19,19 +19,22 @@ def call_hook(sender, instance, **kwargs):
f = getattr(m, func)
except (ValueError, ImportError, AttributeError):
logger.error(
_("malformed return hook '%(hook)s' for [%(name)s]") % {'hook': instance.hook, 'name': instance.name}
_("malformed return hook '%(hook)s' for [%(name)s]")
% {"hook": instance.hook, "name": instance.name}
)
return
try:
f(instance)
except Exception as e:
logger.error(
_(
"return hook %(hook)s failed on [%(name)s] because %(error)s"
) % {'hook': instance.hook, 'name': instance.name, 'error': str(e)}
_("return hook %(hook)s failed on [%(name)s] because %(error)s")
% {"hook": instance.hook, "name": instance.name, "error": str(e)}
)
# args: proc_name
post_spawn = Signal()
# args: task
pre_enqueue = Signal()
@@ -40,3 +43,6 @@ pre_execute = Signal()
# args: task
post_execute = Signal()
# args: func, task
post_execute_in_worker = Signal()

View File

@@ -1,4 +1,5 @@
"""Package signing."""
import pickle
from django_q import core_signing as signing

View File

@@ -1,4 +1,5 @@
"""Provides task functionality."""
# Standard
from multiprocessing import Value
from time import sleep, time
@@ -31,6 +32,7 @@ def async_task(func, *args, **kwargs):
"iter_cached",
"chain",
"broker",
"cluster",
"timeout",
)
q_options = keywords.pop("q_options", {})
@@ -52,7 +54,7 @@ def async_task(func, *args, **kwargs):
elif key in keywords:
task[key] = keywords.pop(key)
# don't serialize the broker
broker = task.pop("broker", get_broker())
broker = task.pop("broker", None) or get_broker(task.get("cluster"))
# overrides
if "cached" not in task and Conf.CACHED:
task["cached"] = Conf.CACHED
@@ -71,7 +73,7 @@ def async_task(func, *args, **kwargs):
return _sync(pack)
# push it
enqueue_id = broker.enqueue(pack)
logger.info(f"Enqueued {enqueue_id}")
logger.info(f"Enqueued [{broker.list_key}] {enqueue_id}")
logger.debug(f"Pushed {tag}")
return task["id"]
@@ -90,6 +92,7 @@ def schedule(func, *args, **kwargs):
:type next_run: datetime.datetime
:param cluster: optional cluster name.
:param cron: optional cron expression
:param intended_date_kwarg: optional identifier to pass intended schedule date.
:param kwargs: function keyword arguments.
:return: the schedule object.
:rtype: Schedule
@@ -102,6 +105,7 @@ def schedule(func, *args, **kwargs):
next_run = kwargs.pop("next_run", timezone.now())
cron = kwargs.pop("cron", None)
cluster = kwargs.pop("cluster", None)
intended_date_kwarg = kwargs.pop("intended_date_kwarg", None)
# check for name duplicates instead of am unique constraint
if name and Schedule.objects.filter(name=name).exists():
@@ -120,6 +124,7 @@ def schedule(func, *args, **kwargs):
next_run=next_run,
cron=cron,
cluster=cluster,
intended_date_kwarg=intended_date_kwarg,
)
# make sure we trigger validation
s.full_clean()
@@ -144,7 +149,7 @@ def result(task_id, wait=0, cached=Conf.CACHED):
start = time()
while True:
r = Task.get_result(task_id)
if r:
if r is not None:
return r
if (time() - start) * 1000 >= wait >= 0:
break
@@ -270,6 +275,7 @@ def fetch_cached(task_id, wait=0, broker=None):
hook=task.get("hook"),
args=task["args"],
kwargs=task["kwargs"],
cluster=task.get("cluster"),
started=task["started"],
stopped=task["stopped"],
result=task["result"],
@@ -340,6 +346,7 @@ def fetch_group_cached(group_id, failures=True, wait=0, count=None, broker=None)
hook=task.get("hook"),
args=task["args"],
kwargs=task["kwargs"],
cluster=task.get("cluster"),
started=task["started"],
stopped=task["stopped"],
result=task["result"],
@@ -454,7 +461,9 @@ def async_iter(func, args_iter, **kwargs):
# save the original arguments
broker = options["broker"]
broker.cache.set(
f"{broker.list_key}:{iter_group}:args", SignedPackage.dumps(args_iter)
f"{broker.list_key}:{iter_group}:args",
SignedPackage.dumps(args_iter),
timeout=None,
)
for args in args_iter:
if not isinstance(args, tuple):
@@ -563,10 +572,12 @@ class Chain:
A sequential chain of tasks
"""
def __init__(self, chain=None, group=None, cached=Conf.CACHED, sync=Conf.SYNC):
def __init__(
self, chain=None, group=None, cached=Conf.CACHED, sync=Conf.SYNC, broker=None
):
self.chain = chain or []
self.group = group or ""
self.broker = get_broker()
self.broker = broker or get_broker()
self.cached = cached
self.sync = sync
self.started = False
@@ -600,7 +611,8 @@ class Chain:
def result(self, wait=0):
"""
return the full list of results from the chain when it finishes. blocks until timeout.
return the full list of results from the chain when it finishes. blocks until
timeout.
:param int wait: how many milliseconds to wait for a result
:return: an unsorted list of results
"""
@@ -611,7 +623,8 @@ class Chain:
def fetch(self, failures=True, wait=0):
"""
get the task result objects from the chain when it finishes. blocks until timeout.
get the task result objects from the chain when it finishes. blocks until
timeout.
:param failures: include failed tasks
:param int wait: how many milliseconds to wait for a result
:return: an unsorted list of task objects
@@ -721,17 +734,14 @@ class AsyncTask:
return self.id
def result(self, wait=0):
if self.started:
return result(self.id, wait=wait, cached=self.cached)
def fetch(self, wait=0):
if self.started:
return fetch(self.id, wait=wait, cached=self.cached)
def result_group(self, failures=False, wait=0, count=None):
if self.started and self.group:
return result_group(
self.group,
@@ -742,7 +752,6 @@ class AsyncTask:
)
def fetch_group(self, failures=True, wait=0, count=None):
if self.started and self.group:
return fetch_group(
self.group,
@@ -755,7 +764,8 @@ class AsyncTask:
def _sync(pack):
"""Simulate a package travelling through the cluster."""
from django_q.cluster import monitor, worker
from django_q.monitor import monitor
from django_q.worker import worker
task_queue = Queue()
result_queue = Queue()

View File

@@ -1,7 +1,5 @@
import os
import django
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
@@ -77,7 +75,7 @@ DATABASES = {
LANGUAGE_CODE = "en-us"
TIME_ZONE = "UTC"
TIME_ZONE = "Europe/Amsterdam"
USE_I18N = True
@@ -118,7 +116,7 @@ CACHES = {
"LOCATION": f"redis://{REDIS_HOST}:6379/0",
"OPTIONS": {
"CLIENT_CLASS": "django_redis.client.DefaultClient",
"PARSER_CLASS": "redis.connection.HiredisParser",
"PARSER_CLASS": "redis.connection.DefaultParser",
},
}
}
@@ -130,5 +128,5 @@ Q_CLUSTER = {
"testing": True,
"log_level": "DEBUG",
"django_redis": "default",
"redis": f"redis://{REDIS_HOST}:6379/0"
"redis": f"redis://{REDIS_HOST}:6379/0",
}

View File

@@ -46,6 +46,10 @@ def hello():
return "hello"
def return_falsy_value():
return []
def result(obj):
print(f"RESULT HOOK {obj.name} : {obj.result()}")

View File

@@ -64,7 +64,7 @@ def test_admin_views(admin_client, monkeypatch):
# resubmit the failure
url = reverse("admin:django_q_failure_changelist")
data = {"action": "retry_failed", "_selected_action": [f.pk]}
data = {"action": "resubmit_task", "_selected_action": [f.pk]}
response = admin_client.post(url, data)
assert response.status_code == 302
assert Failure.objects.filter(name=f.id).exists() is False
@@ -84,3 +84,10 @@ def test_admin_views(admin_client, monkeypatch):
data = {"post": "yes"}
response = admin_client.post(url, data)
assert response.status_code == 302
# Resubmit a successful task.
url = reverse("admin:django_q_success_changelist")
data = {"action": "resubmit_task", "_selected_action": [t.pk]}
initial_queue_count = OrmQ.objects.count()
response = admin_client.post(url, data)
assert response.status_code == 302
assert OrmQ.objects.count() > initial_queue_count

View File

@@ -2,12 +2,11 @@ import os
from time import sleep
import pytest
import redis
from django_q.brokers import Broker, get_broker
from django_q.conf import Conf
from django_q.humanhash import uuid
from django_q.tests.settings import REDIS_HOST, MONGO_HOST
from django_q.tests.settings import MONGO_HOST, REDIS_HOST
def test_broker(monkeypatch):
@@ -36,6 +35,57 @@ def test_broker(monkeypatch):
assert broker.get_stats("test:*") is None
def test_broker_set_stat_prunes_stale_master_list():
# regression: set_stat should prune stale master-list entries on register
broker = Broker()
a_key = f"{Conf.Q_STAT}:A"
b_key = f"{Conf.Q_STAT}:B"
broker.cache.delete(Conf.Q_STAT)
broker.set_stat(a_key, "state_a", 3)
assert a_key in broker.cache.get(Conf.Q_STAT)
# Drop A's per-stat value to simulate a dead cluster whose TTL expired,
# leaving the master-list entry as a stale reference.
broker.cache.delete(a_key)
assert broker.get_stat(a_key) is None
assert a_key in broker.cache.get(Conf.Q_STAT)
# Registering B should prune stale A from the master list.
broker.set_stat(b_key, "state_b", 3)
key_list = broker.cache.get(Conf.Q_STAT)
assert a_key not in key_list
assert b_key in key_list
assert broker.get_stat(b_key) == "state_b"
def test_broker_set_stat_skips_master_list_write_on_repeat(monkeypatch):
# set_stat should only write the master list when membership changes
broker = Broker()
a_key = f"{Conf.Q_STAT}:A"
broker.cache.delete(Conf.Q_STAT)
writes = []
orig_set = broker.cache.set
def counting_set(key, value, timeout=None, **kw):
if key == Conf.Q_STAT:
writes.append(key)
return orig_set(key, value, timeout, **kw)
monkeypatch.setattr(broker.cache, "set", counting_set)
# First call adds a new entry, one master-list write expected
broker.set_stat(a_key, "state_a", 3)
assert len(writes) == 1
# Subsequent calls for the same key must not rewrite the master list
broker.set_stat(a_key, "state_a", 3)
broker.set_stat(a_key, "state_a", 3)
broker.set_stat(a_key, "state_a", 3)
assert len(writes) == 1
def test_redis(monkeypatch):
monkeypatch.setattr(Conf, "DJANGO_REDIS", None)
broker = get_broker()
@@ -52,7 +102,7 @@ def test_redis(monkeypatch):
def test_custom(monkeypatch):
monkeypatch.setattr(Conf, "BROKER_CLASS", "brokers.redis_broker.Redis")
monkeypatch.setattr(Conf, "BROKER_CLASS", "django_q.brokers.redis_broker.Redis")
broker = get_broker()
assert broker.ping() is True
assert broker.info() is not None
@@ -125,7 +175,7 @@ def test_ironmq(monkeypatch):
@pytest.mark.skipif(
not os.getenv("AWS_ACCESS_KEY_ID"), reason="requires AWS credentials"
)
def canceled_sqs(monkeypatch):
def test_sqs(monkeypatch):
monkeypatch.setattr(
Conf,
"SQS",
@@ -133,11 +183,13 @@ def canceled_sqs(monkeypatch):
"aws_region": os.getenv("AWS_REGION"),
"aws_access_key_id": os.getenv("AWS_ACCESS_KEY_ID"),
"aws_secret_access_key": os.getenv("AWS_SECRET_ACCESS_KEY"),
"receive_message_wait_time_seconds": 20,
"receive_message_wait_time_seconds": 5,
},
)
# check broker
broker = get_broker(list_key=uuid()[0])
broker = get_broker(list_key="testing")
assert "receive_message_wait_time_seconds" in Conf.SQS
assert "aws_region" in Conf.SQS
assert broker.ping() is True
assert broker.info() is not None
assert broker.queue_size() == 0
@@ -174,7 +226,7 @@ def canceled_sqs(monkeypatch):
broker.enqueue("test")
while task is None:
task = broker.dequeue()[0]
broker.fail(task[0])
broker.fail(task[0][0])
# bulk test
for _ in range(10):
broker.enqueue("test")

View File

@@ -1,10 +1,13 @@
import time
from copy import deepcopy
from multiprocessing import Event, Value
import pytest
from django_q.brokers import get_broker
from django_q.cluster import monitor, pusher, worker
from django_q.conf import Conf
from django_q.monitor import monitor
from django_q.pusher import pusher
from django_q.queues import Queue
from django_q.tasks import (
AsyncTask,
@@ -21,6 +24,7 @@ from django_q.tasks import (
result,
result_group,
)
from django_q.worker import worker
@pytest.fixture
@@ -126,6 +130,27 @@ def test_iter(broker):
assert len(i.result()) == 5
def _sleeping_func(_):
time.sleep(1)
@pytest.mark.django_db
def test_iter_default_cache_timeout(broker, settings):
"""
Test async_iter when it completes after the default Django
cache timeout expires.
"""
cache_settings = deepcopy(settings.CACHES)
cache_settings["default"]["TIMEOUT"] = 1
settings.CACHES = cache_settings
broker.purge_queue()
broker.cache.clear()
it = [i for i in range(2)]
t = async_iter(_sleeping_func, it, sync=True)
result_t = result(t)
assert result_t is not None
@pytest.mark.django_db
def test_chain(broker):
broker.purge_queue()

View File

@@ -1,8 +1,8 @@
from datetime import datetime
import os
import sys
import threading
import uuid as uuidlib
from datetime import datetime
from math import copysign
from multiprocessing import Event, Value
from time import sleep
@@ -11,16 +11,20 @@ from typing import Optional
import pytest
from django.utils import timezone
myPath = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, myPath + "/../")
from django_q.brokers import Broker, get_broker
from django_q.cluster import Cluster, Sentinel, monitor, pusher, save_task, worker
from django_q.cluster import Cluster, Sentinel
from django_q.conf import Conf
from django_q.humanhash import DEFAULT_WORDLIST, uuid
from django_q.models import Success, Task
from django_q.monitor import monitor, save_task
from django_q.pusher import pusher
from django_q.queues import Queue
from django_q.signals import post_execute, pre_enqueue, pre_execute
from django_q.signals import (
post_execute,
post_execute_in_worker,
pre_enqueue,
pre_execute,
)
from django_q.status import Stat
from django_q.tasks import (
async_task,
@@ -34,6 +38,10 @@ from django_q.tasks import (
)
from django_q.tests.tasks import TaskError, multiply
from django_q.utils import add_months, add_years
from django_q.worker import worker
myPath = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, myPath + "/../")
class WordClass:
@@ -64,7 +72,7 @@ def test_sync(broker):
@pytest.mark.django_db
def test_sync_raise_exception(broker):
with pytest.raises(Exception):
with pytest.raises(TaskError):
async_task("django_q.tests.tasks.raise_exception", broker=broker, sync=True)
@@ -138,6 +146,30 @@ def test_cluster(broker):
broker.delete_queue()
@pytest.mark.django_db
def test_results(broker):
broker.list_key = "cluster_test:q"
broker.delete_queue()
a = async_task(
"django_q.tests.tasks.return_falsy_value",
broker=broker,
)
task_queue = Queue()
stop_event = Event()
stop_event.set()
pusher(task_queue, stop_event, broker=broker)
task_queue.put("STOP")
result_queue = Queue()
worker(task_queue, result_queue, Value("f", -1))
result_queue.put("STOP")
monitor(result_queue)
# should not loop indefinitely when a real value is returned
value = result(a, wait=-1)
assert value == []
@pytest.mark.django_db
def test_enqueue(broker, admin_user):
broker.list_key = "cluster_test:q"
@@ -372,7 +404,7 @@ def test_timeout_task_finishes(broker, cluster_config_timeout, async_task_kwargs
@pytest.mark.django_db
def test_recycle(broker, monkeypatch):
def test_recycle(broker, monkeypatch, django_assert_num_queries):
# set up the Sentinel
broker.list_key = "test_recycle_test:q"
async_task("django_q.tests.tasks.multiply", 2, 2, broker=broker)
@@ -405,10 +437,12 @@ def test_recycle(broker, monkeypatch):
monkeypatch.setattr(Conf, "SAVE_LIMIT", 1)
result_queue.put("STOP")
# run monitor
monitor(result_queue)
with django_assert_num_queries(12):
monitor(result_queue)
assert Success.objects.count() == Conf.SAVE_LIMIT
broker.delete_queue()
@pytest.mark.django_db
def test_save_limit_per_func(broker, monkeypatch):
# set up the Sentinel
@@ -442,15 +476,14 @@ def test_save_limit_per_func(broker, monkeypatch):
# run monitor
monitor(result_queue)
assert Success.objects.count() == 3
assert set(Success.objects.filter().values_list('func', flat=True)) == {
'django_q.tests.tasks.countdown',
'django_q.tests.tasks.hello',
'django_q.tests.tasks.multiply',
assert set(Success.objects.filter().values_list("func", flat=True)) == {
"django_q.tests.tasks.countdown",
"django_q.tests.tasks.hello",
"django_q.tests.tasks.multiply",
}
broker.delete_queue()
@pytest.mark.django_db
def test_max_rss(broker, monkeypatch):
# set up the Sentinel
@@ -460,7 +493,7 @@ def test_max_rss(broker, monkeypatch):
stop_event = Event()
cluster_id = uuidlib.uuid4()
# override settings
monkeypatch.setattr(Conf, "MAX_RSS", 40000)
monkeypatch.setattr(Conf, "MAX_RSS", 20000)
monkeypatch.setattr(Conf, "WORKERS", 1)
# set a timer to stop the Sentinel
threading.Timer(3, stop_event.set).start()
@@ -538,7 +571,6 @@ def test_attempt_count(broker, monkeypatch):
assert saved_task.attempt_count == 1
sleep(0.5)
# second save
old_stopped = task["stopped"]
task["stopped"] = timezone.now()
save_task(task, broker)
saved_task = Task.objects.get(id=task["id"])
@@ -733,6 +765,35 @@ class TestSignals:
assert self.task.get("result") == -1
post_execute.disconnect(handler)
@pytest.mark.django_db
def test_post_execute_in_worker_signal(self, broker):
broker.list_key = "post_execute_in_worker_test:q"
broker.delete_queue()
self.signal_was_called: bool = False
self.task: Optional[dict] = None
self.func = None
def handler(sender, task, **kwargs):
self.signal_was_called = True
self.task = task
post_execute_in_worker.connect(handler)
task_id = async_task("math.copysign", 1, -1, broker=broker)
task_queue = Queue()
result_queue = Queue()
event = Event()
event.set()
pusher(task_queue, event, broker=broker)
task_queue.put("STOP")
worker(task_queue, result_queue, Value("f", -1))
result_queue.put("STOP")
monitor(result_queue, broker)
broker.delete_queue()
assert self.signal_was_called is True
assert self.task.get("id") == task_id
assert self.task.get("result") == -1
post_execute_in_worker.disconnect(handler)
@pytest.mark.django_db
def assert_result(task):
@@ -770,6 +831,7 @@ def test_add_months():
assert new_date.month == 2
assert new_date.day == 29
@pytest.mark.django_db
def test_add_years():
# add some months

View File

@@ -5,7 +5,7 @@ import pytest
from django_q.brokers import get_broker
from django_q.cluster import Cluster
from django_q.conf import Conf
from django_q.monitor import get_ids, info, monitor
from django_q.monitor_terminal import get_ids, info, monitor
from django_q.status import Stat
from django_q.tasks import async_task

View File

@@ -1,8 +1,9 @@
import os
from datetime import timedelta
from datetime import datetime, timedelta
from multiprocessing import Event, Value
from unittest import mock
import django
import pytest
from django.core.exceptions import ValidationError
from django.db import IntegrityError
@@ -11,9 +12,11 @@ from django.utils import timezone
from django.utils.timezone import is_naive
from django_q.brokers import Broker, get_broker
from django_q.cluster import monitor, pusher, scheduler, worker, localtime
from django_q.conf import Conf
from django_q.monitor import monitor
from django_q.pusher import pusher
from django_q.queues import Queue
from django_q.scheduler import scheduler
from django_q.tasks import Schedule, fetch
from django_q.tasks import schedule as create_schedule
from django_q.tests.settings import BASE_DIR
@@ -21,12 +24,25 @@ from django_q.tests.testing_utilities.multiple_database_routers import (
TestingMultipleAppsDatabaseRouter,
TestingReplicaDatabaseRouter,
)
from django_q.utils import add_months, add_years
from django_q.utils import add_months, localtime
from django_q.worker import worker
if django.VERSION < (4, 0):
# pytz is the default in django 3.2. Remove when no support for 3.2
from pytz import timezone as ZoneInfo
else:
try:
from zoneinfo import ZoneInfo
except ImportError:
from backports.zoneinfo import ZoneInfo
@pytest.fixture
def broker(monkeypatch) -> Broker:
"""Patches the Conf object setting the DJANGO_REDIS attribute allowing a default redis configuration."""
"""
Patches the Conf object setting the DJANGO_REDIS attribute allowing a default
redis configuration.
"""
monkeypatch.setattr(Conf, "DJANGO_REDIS", "default")
return get_broker()
@@ -37,25 +53,11 @@ def orm_broker(monkeypatch) -> None:
monkeypatch.setattr(Conf, "ORM", "default")
@pytest.fixture
def orm_no_replica_broker(orm_broker, monkeypatch) -> Broker:
"""Generates a Broker with a disabled read replica database configuration."""
monkeypatch.setattr(Conf, "HAS_REPLICA", False)
return get_broker(list_key="scheduler_test:q")
@pytest.fixture
def orm_replica_broker(orm_broker, monkeypatch) -> Broker:
"""Generates a Broker with read replica database configuration."""
monkeypatch.setattr(Conf, "HAS_REPLICA", True)
return get_broker(list_key="scheduler_test:q")
REPLICA_DATABASE_ROUTERS = [
f"{TestingReplicaDatabaseRouter.__module__}.{TestingReplicaDatabaseRouter.__name__}"
]
REPLICA_DATABASES = {
"default": {
"writable": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": os.path.join(BASE_DIR, "db.sqlite3"),
},
@@ -66,7 +68,7 @@ REPLICA_DATABASES = {
}
MULTIPLE_APPS_DATABASE_ROUTERS = [
f"{TestingMultipleAppsDatabaseRouter.__module__}.{TestingMultipleAppsDatabaseRouter.__name__}"
f"{TestingMultipleAppsDatabaseRouter.__module__}.{TestingMultipleAppsDatabaseRouter.__name__}" # noqa: E501
]
MULTIPLE_APPS_DATABASES = {
"default": {
@@ -80,6 +82,108 @@ MULTIPLE_APPS_DATABASES = {
}
@pytest.mark.django_db
def test_scheduler_daylight_saving_time_daily(broker, monkeypatch):
# Set up a startdate in the Amsterdam timezone (without dst 1 hour ahead). The
# 28th of March 2021 is the day when sunlight saving starts (at 2 am)
monkeypatch.setattr(Conf, "TIME_ZONE", "Europe/Amsterdam")
tz = ZoneInfo("Europe/Amsterdam")
broker.list_key = "scheduler_test:q"
# Let's start a schedule at 1 am on the 27th of March. This is in AMS timezone.
# So, 2021-03-27 00:00:00 when saved (due to TZ being Amsterdam and saved in UTC)
start_date = datetime(2021, 3, 27, 1, 0, 0)
# Create schedule with the next run date on the start date. It will move one day
# forward when we run the scheduler
schedule = create_schedule(
"math.copysign",
1,
-1,
name="test math",
schedule_type=Schedule.DAILY,
next_run=start_date,
)
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
# It's now the day after exactly at midnight UTC
next_run = schedule.next_run
assert str(next_run) == "2021-03-28 00:00:00+00:00"
# In the Amsterdam timezone, it's 1 hour over midnight (+01)
next_run = next_run.astimezone(tz)
assert str(next_run) == "2021-03-28 01:00:00+01:00"
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
next_run = schedule.next_run
assert str(next_run) == "2021-03-28 23:00:00+00:00"
next_run = next_run.astimezone(tz)
# In the Amsterdam timezone, it's 1 hour over midnight (+02)
assert str(next_run) == "2021-03-29 01:00:00+02:00"
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
next_run = schedule.next_run
assert str(next_run) == "2021-03-29 23:00:00+00:00"
next_run = next_run.astimezone(tz)
assert str(next_run) == "2021-03-30 01:00:00+02:00"
# Create second schedule with the next run date on the start date. It will move
# one day forward when we run the scheduler
start_date = datetime(2021, 10, 29, 1, 0, 0)
schedule = create_schedule(
"django_q.tests.tasks.word_multiply",
2,
name="multiply",
schedule_type=Schedule.DAILY,
next_run=start_date,
)
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
next_run = schedule.next_run
assert str(next_run) == "2021-10-29 23:00:00+00:00"
# In the Amsterdam timezone, it's 1 hour over midnight (+02)
next_run = next_run.astimezone(tz)
assert str(next_run) == "2021-10-30 01:00:00+02:00"
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
next_run = schedule.next_run
assert str(next_run) == "2021-10-30 23:00:00+00:00"
# In the Amsterdam timezone, it's 1 hour over midnight (+02)
next_run = next_run.astimezone(tz)
assert str(next_run) == "2021-10-31 01:00:00+02:00"
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
next_run = schedule.next_run
assert str(next_run) == "2021-11-01 00:00:00+00:00"
# In the Amsterdam timezone, it's 1 hour over midnight (+01)
# Switch of DST
next_run = next_run.astimezone(tz)
assert str(next_run) == "2021-11-01 01:00:00+01:00"
@pytest.mark.django_db
def test_scheduler(broker, monkeypatch):
broker.list_key = "scheduler_test:q"
@@ -234,7 +338,7 @@ def test_scheduler(broker, monkeypatch):
"django_q.tests.tasks.word_multiply",
2,
word="catch_up",
schedule_type=Schedule.BIMONTHLY
schedule_type=Schedule.BIMONTHLY,
)
scheduler(broker=broker)
schedule = Schedule.objects.get(pk=schedule.pk)
@@ -245,14 +349,14 @@ def test_scheduler(broker, monkeypatch):
"django_q.tests.tasks.word_multiply",
2,
word="catch_up",
schedule_type=Schedule.BIWEEKLY
schedule_type=Schedule.BIWEEKLY,
)
scheduler(broker=broker)
schedule = Schedule.objects.get(pk=schedule.pk)
assert schedule.next_run.date() == (timezone.now() + timedelta(weeks=2)).date()
broker.delete_queue()
monkeypatch.setattr(Conf, "PREFIX", "some_cluster_name")
monkeypatch.setattr(Conf, "CLUSTER_NAME", "some_cluster_name")
# create a schedule on another cluster
schedule = create_schedule(
"math.copysign",
@@ -276,7 +380,7 @@ def test_scheduler(broker, monkeypatch):
# queue must be empty
assert task_queue.qsize() == 0
monkeypatch.setattr(Conf, "PREFIX", "default")
monkeypatch.setattr(Conf, "CLUSTER_NAME", "default")
# create a schedule on the same cluster
schedule = create_schedule(
"math.copysign",
@@ -301,61 +405,113 @@ def test_scheduler(broker, monkeypatch):
assert task_queue.qsize() == 1
@override_settings(
DATABASE_ROUTERS=REPLICA_DATABASE_ROUTERS, DATABASES=REPLICA_DATABASES
)
@pytest.mark.django_db
def test_scheduler_atomic_transaction_must_specify_a_database_when_no_replicas_are_used(
orm_no_replica_broker: Broker,
):
"""
GIVEN a environment without a read replica database
WHEN the scheduler is called
THEN the transaction atomic must be called using the configured database in the Conf.ORM settings.
"""
broker = orm_no_replica_broker
with mock.patch("django_q.cluster.db") as mocked_db:
scheduler(broker=broker)
# The router should correctly set the database to use!
mocked_db.transaction.atomic.assert_called_with(using=broker.connection.db)
def test_intended_schedule_kwarg(broker, monkeypatch):
broker.list_key = "scheduler_test:q"
broker.delete_queue()
run_date = timezone.now() - timedelta(hours=1)
schedule = create_schedule(
"math.copysign",
1,
-1,
name="test math",
hook="django_q.tests.tasks.result",
schedule_type=Schedule.HOURLY,
repeats=1,
next_run=run_date,
intended_date_kwarg="intended_date",
)
assert schedule.last_run() is None
assert schedule.intended_date_kwarg == "intended_date"
# run scheduler
scheduler(broker=broker)
# set up the workflow
task_queue = Queue()
stop_event = Event()
stop_event.set()
# push it
pusher(task_queue, stop_event, broker=broker)
assert task_queue.qsize() == 1
task = task_queue.get()
assert "intended_date" in task["kwargs"]
assert task["kwargs"]["intended_date"] == run_date.isoformat()
@override_settings(
DATABASE_ROUTERS=REPLICA_DATABASE_ROUTERS, DATABASES=REPLICA_DATABASES
)
@pytest.mark.django_db
def test_scheduler_atomic_transaction_must_specify_no_database_when_read_write_replicas_are_used(
orm_replica_broker: Broker,
def test_scheduler_atomic_must_specify_the_write_db(
orm_broker: Broker,
):
"""
GIVEN a environment with a read/write configured replica database
WHEN the scheduler is called
THEN the transaction must be called without a specific database, thus letting the database router pick.
THEN the transaction must be called with the write database.
"""
with mock.patch("django_q.cluster.db") as mocked_db:
scheduler(broker=orm_replica_broker)
# No specific databases should be set here, this is the job of the router!
mocked_db.transaction.atomic.assert_called_with()
broker = get_broker(list_key="scheduler_test:q")
with mock.patch("django_q.cluster.db.transaction") as mocked_db:
scheduler(broker=broker)
mocked_db.atomic.assert_called_with(using="writable")
@override_settings(
DATABASE_ROUTERS=MULTIPLE_APPS_DATABASE_ROUTERS, DATABASES=MULTIPLE_APPS_DATABASES
)
@pytest.mark.django_db
def test_scheduler_atomic_transaction_must_specify_the_database_based_on_router_redirection(
orm_no_replica_broker: Broker,
def test_scheduler_atomic_must_specify_the_database_based_on_router_redirection(
orm_broker: Broker,
):
"""
GIVEN a environment without a read replica database
WHEN the scheduler is called
THEN the transaction atomic must be called using the configured database in the Conf.ORM settings.
THEN the transaction atomic must be called using the default connection.
"""
broker = orm_no_replica_broker
with mock.patch("django_q.cluster.db") as mocked_db:
broker = get_broker(list_key="scheduler_test:q")
with mock.patch("django_q.cluster.db.transaction") as mocked_db:
scheduler(broker=broker)
# The router should correctly set the database to use!
assert broker.connection.db == "default"
mocked_db.transaction.atomic.assert_called_with(using=broker.connection.db)
mocked_db.atomic.assert_called_with(using="default")
@pytest.mark.django_db
def test_schedule_save_sets_next_run_for_cron():
"""Ensure Schedule.save() sets next_run correctly for CRON schedules."""
cron_expression = "0 12 * * *" # Executes dialy at 12pm
schedule = Schedule(
func="math.sqrt",
schedule_type=Schedule.CRON,
cron=cron_expression,
)
assert schedule.next_run is not None
assert schedule.pk is None
initial_next_run = schedule.next_run
schedule.save()
# After save, next_run must be recalculated based on the CRON expression
assert schedule.next_run > initial_next_run
@pytest.mark.django_db
def test_schedule_save_direct_db(broker):
"""Ensure Schedule.save() updates next_run correctly when created directly in DB."""
cron_expression = "0 12 * * *" # Executes dialy at 12pm
# Creating schedule directly in database
schedule = Schedule.objects.create(
func="math.sqrt",
schedule_type=Schedule.CRON,
cron=cron_expression,
)
# Next run must be defined after execution time
assert schedule.next_run is not None
assert schedule.next_run > timezone.now()
scheduler(broker)
assert broker.queue_size() == 0
def test_localtime():

View File

@@ -12,9 +12,9 @@ class TestingReplicaDatabaseRouter:
def db_for_write(self, model, **hints):
"""
Always write to DEFAULT database
Always write to WRITABLE database
"""
return "default"
return "writable"
class TestingMultipleAppsDatabaseRouter:

43
django_q/timeout.py Normal file
View File

@@ -0,0 +1,43 @@
import signal
from django.utils.translation import gettext_lazy as _
from django_q.conf import logger
from .exceptions import TimeoutException
class TimeoutHandler:
def __init__(self, timeout: int):
self._timeout = timeout
def raise_timeout_exception(self, signum, frame):
raise TimeoutException(
f"Task exceeded maximum timeout value ({self._timeout} seconds)"
)
def __enter__(self):
# if the timeout is -1, then there is no timeout and the task will always keep running until it's done or manually killed
if self._timeout == -1:
return
try:
signal.signal(signal.SIGALRM, self.raise_timeout_exception)
signal.alarm(self._timeout)
except (
ValueError,
AttributeError,
): # AttributeError or ValueError might be raised for Windows users
logger.debug(_("SIGALARM is not available on your platform"))
def __exit__(self, exc_type, exc_value, traceback):
if self._timeout == -1:
return
"""When getting out of the timeout, reset the alarm, so it won't trigger"""
try:
signal.alarm(0)
signal.signal(signal.SIGALRM, signal.SIG_DFL)
except (
ValueError,
AttributeError,
): # AttributeError or ValueError might be raised for Windows users
logger.debug(_("SIGALARM is not available on your platform"))

View File

@@ -1,6 +1,23 @@
import inspect
from datetime import date
import calendar
import inspect
from datetime import date, datetime
import django
from django import db
from django.conf import settings
from django.utils import timezone
from django_q.conf import Conf, logger
if django.VERSION < (4, 0):
# pytz is the default in django 3.2. Remove when no support for 3.2
from pytz import timezone as ZoneInfo
else:
try:
from zoneinfo import ZoneInfo
except ImportError:
from backports.zoneinfo import ZoneInfo
# credits: https://stackoverflow.com/a/4131114
# Made them aware of timezone
@@ -8,21 +25,21 @@ def add_months(d, months):
month = d.month - 1 + months
year = d.year + month // 12
month = month % 12 + 1
day = min(d.day, calendar.monthrange(year,month)[1])
day = min(d.day, calendar.monthrange(year, month)[1])
return d.replace(year=year, month=month, day=day)
# credits: https://stackoverflow.com/a/15743908
# Changed the last line to make it a little easier to read and changed it to move February 29 to 28 next year
# Also made them aware of timezone
# Changed the last line to make it a little easier to read and changed it to move
# February 29 to 28 next year.
def add_years(d, years):
"""Return a date that's `years` years after the date (or datetime)
object `d`. Return the same calendar date (month and day) in the
destination year, if it exists, otherwise use the previous day
(thus changing February 29 to February 28).
"""
try:
return d.replace(year = d.year + years)
return d.replace(year=d.year + years)
except ValueError:
new_date = d + (date(d.year + years, 3, 1) - date(d.year, 3, 1))
return d.replace(year=new_date.year, month=new_date.month, day=new_date.day)
@@ -32,11 +49,38 @@ def get_func_repr(func):
# convert func to string
if inspect.isfunction(func):
return f"{func.__module__}.{func.__name__}"
elif inspect.ismethod(func) and hasattr(func.__self__, '__name__'):
return (
f"{func.__self__.__module__}."
f"{func.__self__.__name__}.{func.__name__}"
elif inspect.ismethod(func) and hasattr(func.__self__, "__name__"):
return f"{func.__self__.__module__}.{func.__self__.__name__}.{func.__name__}"
else:
return str(func) if func else None
def localtime(value=None) -> datetime:
"""Override for timezone.localtime to deal with naive times and local times"""
if settings.USE_TZ:
if django.VERSION >= (4, 0) and getattr(settings, "USE_DEPRECATED_PYTZ", False):
import pytz
convert_to_tz = pytz.timezone(Conf.TIME_ZONE)
else:
convert_to_tz = ZoneInfo(Conf.TIME_ZONE)
return timezone.localtime(value=value, timezone=convert_to_tz)
if value is None:
return datetime.now()
else:
return value
def close_old_django_connections():
"""
Close django connections unless running with sync=True.
"""
if Conf.SYNC:
logger.warning(
"Preserving django database connections because sync=True. Beware "
"that tasks are now injected in the calling context/transactions "
"which may result in unexpected behaviour."
)
else:
return str(func)
db.close_old_connections()

144
django_q/worker.py Normal file
View File

@@ -0,0 +1,144 @@
import pydoc
import traceback
from multiprocessing import Value
from multiprocessing.process import current_process
from multiprocessing.queues import Queue
from django import core
from django.apps.registry import apps
from django.utils import timezone
from django.utils.translation import gettext_lazy as _
try:
apps.check_apps_ready()
except core.exceptions.AppRegistryNotReady:
import django
django.setup()
from django_q.conf import Conf, error_reporter, logger, resource, setproctitle
from django_q.exceptions import TimeoutException
from django_q.signals import post_execute_in_worker, post_spawn, pre_execute
from django_q.timeout import TimeoutHandler
from django_q.utils import close_old_django_connections, get_func_repr
try:
import psutil
except ImportError:
psutil = None
try:
import setproctitle
except ModuleNotFoundError:
setproctitle = None
def worker(
task_queue: Queue, result_queue: Queue, timer: Value, timeout: int = Conf.TIMEOUT
):
"""
Takes a task from the task queue, tries to execute it and puts the result back in
the result queue
:param timeout: number of seconds wait for a worker to finish.
:type task_queue: multiprocessing.Queue
:type result_queue: multiprocessing.Queue
:type timer: multiprocessing.Value
"""
proc_name = current_process().name
logger.info(
_("%(proc_name)s ready for work at %(id)s")
% {"proc_name": proc_name, "id": current_process().pid}
)
post_spawn.send(sender="django_q", proc_name=proc_name)
if setproctitle:
setproctitle.setproctitle(f"qcluster {proc_name} idle")
task_count = 0
if timeout is None:
timeout = -1
# Start reading the task queue
for task in iter(task_queue.get, "STOP"):
result = None
timer.value = -1 # Idle
task_count += 1
f = task["func"]
# Log task creation and set process name
# Get the function from the task
func_name = get_func_repr(f)
task_name = task["name"]
task_desc = _("%(proc_name)s processing %(task_name)s '%(func_name)s'") % {
"proc_name": proc_name,
"func_name": func_name,
"task_name": task_name,
}
if "group" in task:
task_desc += f" [{task['group']}]"
logger.info(task_desc)
if setproctitle:
proc_title = f"qcluster {proc_name} processing {task_name} '{func_name}'"
if "group" in task:
proc_title += f" [{task['group']}]"
setproctitle.setproctitle(proc_title)
# if it's not an instance try to get it from the string
if not callable(f):
# locate() returns None if f cannot be loaded
f = pydoc.locate(f)
if not task.get("sync", False):
close_old_django_connections()
timer_value = task.pop("timeout", timeout)
# signal execution
pre_execute.send(sender="django_q", func=f, task=task)
# execute the payload
timer.value = timer_value # Busy
if timer.value != -1:
timer.value += 3 # Add buffer so that guard doesn't kill the process on timeout before it gets processed
timeout_error = False
try:
if f is None:
# raise a meaningfull error if task["func"] is not a valid function
raise ValueError(f"Function {task['func']} is not defined")
with TimeoutHandler(timer_value):
res = f(*task["args"], **task["kwargs"])
result = (res, True)
except (Exception, TimeoutException) as e:
if isinstance(e, TimeoutException):
timeout_error = True
result = (f"{e} : {traceback.format_exc()}", False)
if error_reporter:
error_reporter.report()
if task.get("sync", False):
post_execute_in_worker.send(sender="django_q", func=f, task=task)
raise
with timer.get_lock():
# Process result
task["result"] = result[0]
task["success"] = result[1]
task["stopped"] = timezone.now()
post_execute_in_worker.send(sender="django_q", func=f, task=task)
result_queue.put(task)
if timeout_error:
# force destroy process due to timeout
timer.value = 0
break
timer.value = -1 # Idle
if setproctitle:
setproctitle.setproctitle(f"qcluster {proc_name} idle")
# Recycle
if task_count == Conf.RECYCLE or rss_check():
timer.value = -2 # Recycled
break
logger.info(_("%(proc_name)s stopped doing work") % {"proc_name": proc_name})
def rss_check():
if Conf.MAX_RSS:
if resource:
return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss >= Conf.MAX_RSS
elif psutil:
return psutil.Process().memory_info().rss >= Conf.MAX_RSS * 1024
return False

View File

@@ -49,8 +49,9 @@ Reference
:param str group: an optional group name.
:param bool cached: run this against the cache backend
:param bool sync: execute this inline instead of asynchronous
:param broker: an optional broker instance
.. py:class:: Chain(chain=None, group=None, cached=Conf.CACHED, sync=Conf.SYNC)
.. py:class:: Chain(chain=None, group=None, cached=Conf.CACHED, sync=Conf.SYNC, broker=None)
A sequential chain of tasks. Acts as a convenient wrapper for :func:`async_chain`
You can pass the task chain at construction or you can append individual tasks before running them.
@@ -59,6 +60,8 @@ Reference
:param str group: an optional group name.
:param bool cached: run this against the cache backend
:param bool sync: execute this inline instead of asynchronous
:param bool sync: execute this inline instead of asynchronous
:param broker: an optional broker instance
.. py:method:: append(func, *args, **kwargs)

View File

@@ -11,36 +11,36 @@ Start your cluster using Django's ``manage.py`` command::
You should see the cluster starting ::
10:57:40 [Q] INFO Q Cluster-31781 starting.
10:57:40 [Q] INFO Process-1:1 ready for work at 31784
10:57:40 [Q] INFO Process-1:2 ready for work at 31785
10:57:40 [Q] INFO Process-1:3 ready for work at 31786
10:57:40 [Q] INFO Process-1:4 ready for work at 31787
10:57:40 [Q] INFO Process-1:5 ready for work at 31788
10:57:40 [Q] INFO Process-1:6 ready for work at 31789
10:57:40 [Q] INFO Process-1:7 ready for work at 31790
10:57:40 [Q] INFO Process-1:8 ready for work at 31791
10:57:40 [Q] INFO Process-1:9 monitoring at 31792
10:57:40 [Q] INFO Process-1 guarding cluster at 31783
10:57:40 [Q] INFO Process-1:10 pushing tasks at 31793
10:57:40 [Q] INFO Q Cluster-31781 running.
10:57:40 [Q] INFO Q Cluster freddie-uncle-twenty-ten starting.
10:57:40 [Q] INFO Process-ede257774c4444c980ab479f10947acc ready for work at 31784
10:57:40 [Q] INFO Process-ed580482da3f42968230baa2e4253e42 ready for work at 31785
10:57:40 [Q] INFO Process-8a370dc2bc1d49aa9864e517c9895f74 ready for work at 31786
10:57:40 [Q] INFO Process-74912f9844264d1397c6e54476b530c0 ready for work at 31787
10:57:40 [Q] INFO Process-b00edb26c6074a6189e5696c60aeb35b ready for work at 31788
10:57:40 [Q] INFO Process-b0862965db04479f9784a26639ee51e0 ready for work at 31789
10:57:40 [Q] INFO Process-7e8abbb8ca2d4d9bb20a937dd5e2872b ready for work at 31790
10:57:40 [Q] INFO Process-b0862965db04479f9784a26639ee51e0 ready for work at 31791
10:57:40 [Q] INFO Process-67fa9461ac034736a766cd813f617e62 monitoring at 31792
10:57:40 [Q] INFO Process-eac052c646b2459797cee98bdb84c85d guarding cluster at 31783
10:57:40 [Q] INFO Process-5d98deb19b1e4b2da2ef1e5bd6824f75 pushing tasks at 31793
10:57:40 [Q] INFO Q Cluster freddie-uncle-twenty-ten running.
Stopping the cluster with ctrl-c or either the ``SIGTERM`` and ``SIGKILL`` signals, will initiate the :ref:`stop_procedure`::
16:44:12 [Q] INFO Q Cluster-31781 stopping.
16:44:12 [Q] INFO Process-1 stopping cluster processes
16:44:13 [Q] INFO Process-1:10 stopped pushing tasks
16:44:13 [Q] INFO Process-1:6 stopped doing work
16:44:13 [Q] INFO Process-1:4 stopped doing work
16:44:13 [Q] INFO Process-1:1 stopped doing work
16:44:13 [Q] INFO Process-1:5 stopped doing work
16:44:13 [Q] INFO Process-1:7 stopped doing work
16:44:13 [Q] INFO Process-1:3 stopped doing work
16:44:13 [Q] INFO Process-1:8 stopped doing work
16:44:13 [Q] INFO Process-1:2 stopped doing work
16:44:14 [Q] INFO Process-1:9 stopped monitoring results
16:44:15 [Q] INFO Q Cluster-31781 has stopped.
16:44:12 [Q] INFO Q Cluster freddie-uncle-twenty-ten stopping.
16:44:12 [Q] INFO Process-eac052c646b2459797cee98bdb84c85d stopping cluster processes
16:44:13 [Q] INFO Process-5d98deb19b1e4b2da2ef1e5bd6824f75 stopped pushing tasks
16:44:13 [Q] INFO Process-b0862965db04479f9784a26639ee51e0 stopped doing work
16:44:13 [Q] INFO Process-7e8abbb8ca2d4d9bb20a937dd5e2872b stopped doing work
16:44:13 [Q] INFO Process-b0862965db04479f9784a26639ee51e0 stopped doing work
16:44:13 [Q] INFO Process-b00edb26c6074a6189e5696c60aeb35b stopped doing work
16:44:13 [Q] INFO Process-74912f9844264d1397c6e54476b530c0 stopped doing work
16:44:13 [Q] INFO Process-8a370dc2bc1d49aa9864e517c9895f74 stopped doing work
16:44:13 [Q] INFO Process-ed580482da3f42968230baa2e4253e42 stopped doing work
16:44:13 [Q] INFO Process-ede257774c4444c980ab479f10947acc stopped doing work
16:44:14 [Q] INFO Process-67fa9461ac034736a766cd813f617e62 stopped monitoring results
16:44:15 [Q] INFO Q Cluster freddie-uncle-twenty-ten has stopped.
The number of workers, optional timeouts, recycles and cpu_affinity can be controlled via the :doc:`configure` settings.
@@ -52,6 +52,30 @@ You can have multiple clusters on multiple machines, working on the same queue a
- They use the same cluster name. See :doc:`configure`
- They share the same ``SECRET_KEY`` for Django.
.. _multiple-queues
Multiple Queues
-----------------
You can have multiple queues in one Django site, and use multiple cluster to work on each queue.
Different queues are identified by different queue names which are also cluster names.
To run an alternate cluster, e.g. to work on the 'long' queue, start your cluster with command::
# On Linux
$ Q_CLUSTER_NAME=long python manage.py qcluster
# On Windows
$ python manage.py qcluster --name long
You can set different Q_CLUSTER options for alternative clusters, such as 'timeout', 'queue_limit'
and any other options which are valid in :doc:`configure`. See :ref:`alt-clusters`.
.. note::
To use multiple queue, use the keyword argument `cluster` in async_task() and schedule():
* if `cluster` is not set (the default), async_task() and schedule() will be handled by the default cluster;
* if `cluster` is set, only clusters with matching cluster name will run the task or do the schedule.
Using a Procfile
----------------
If you host on `Heroku <https://heroku.com>`__ or you are using `Honcho <https://github.com/nickstenning/honcho>`__ you can start the cluster from a :file:`Procfile` with an entry like this::

View File

@@ -16,12 +16,10 @@
import os
import sys
import sphinx_rtd_theme
myPath = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, myPath + '/../')
os.environ['DJANGO_SETTINGS_MODULE'] = 'django_q.tests.settings'
nitpick_ignore = [('py:class', 'datetime')]
sys.path.insert(0, myPath + "/../")
os.environ["DJANGO_SETTINGS_MODULE"] = "django_q.tests.settings"
nitpick_ignore = [("py:class", "datetime")]
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
@@ -37,50 +35,54 @@ nitpick_ignore = [('py:class', 'datetime')]
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx_rtd_theme',
'sphinx.ext.todo',
'sphinx.ext.intersphinx',
"sphinx_rtd_theme",
"sphinx.ext.todo",
"sphinx.ext.intersphinx",
# 'sphinx.ext.autodoc'
]
intersphinx_mapping = {'python': ('https://docs.python.org/3.8', None),
'django': ('https://docs.djangoproject.com/en/2.2/',
'https://docs.djangoproject.com/en/2.2/_objects/')}
intersphinx_mapping = {
"python": ("https://docs.python.org/3.8", None),
"django": (
"https://docs.djangoproject.com/en/2.2/",
"https://docs.djangoproject.com/en/2.2/_objects/",
),
}
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
templates_path = ["_templates"]
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
source_suffix = ".rst"
# The encoding of source files.
# source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
master_doc = "index"
# General information about the project.
project = 'Django Q2'
copyright = '2015-2021, Ilan Steemers - 2022, Stan Triepels'
author = 'Ilan Steemers, Stan Triepels'
project = "Django Q2"
copyright = "2015-2021, Ilan Steemers - 2022, Stan Triepels"
author = "Ilan Steemers, Stan Triepels"
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '1.4'
version = "1.10"
# The full version, including alpha/beta/rc tags.
release = '1.4.5'
release = "1.10.0"
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = 'en'
language = "en"
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
@@ -90,7 +92,7 @@ language = 'en'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build']
exclude_patterns = ["_build"]
# The reST default role (used for this markup: `text`) to use for all
# documents.
@@ -108,7 +110,7 @@ add_module_names = False
# show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
pygments_style = "sphinx"
# A list of ignored prefixes for module index sorting.
# modindex_common_prefix = []
@@ -124,7 +126,7 @@ todo_include_todos = True
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'sphinx_rtd_theme'
html_theme = "sphinx_rtd_theme"
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
@@ -136,11 +138,11 @@ html_theme_options = {
# 'github_banner': True,
}
html_sidebars = {
'**': [
'about.html',
'navigation.html',
'relations.html',
'searchbox.html',
"**": [
"about.html",
"navigation.html",
"relations.html",
"searchbox.html",
]
}
# Add any paths that contain custom themes here, relative to this directory.
@@ -161,12 +163,12 @@ html_sidebars = {
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
html_favicon = '_static/favicon.ico'
html_favicon = "_static/favicon.ico"
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
html_static_path = ["_static"]
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
@@ -229,20 +231,17 @@ html_static_path = ['_static']
# html_search_scorer = 'scorer.js'
# Output file base name for HTML help builder.
htmlhelp_basename = 'DjangoQ2doc'
htmlhelp_basename = "DjangoQ2doc"
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
# 'preamble': '',
# Latex figure (float) alignment
# 'figure_align': 'htbp',
}
@@ -251,8 +250,7 @@ latex_elements = {
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'DjangoQ2.tex', 'Django Q2 Documentation',
'Ilan Steemers', 'manual'),
(master_doc, "DjangoQ2.tex", "Django Q2 Documentation", "Ilan Steemers", "manual"),
]
# The name of an image file (relative to this directory) to place at the top of
@@ -280,10 +278,7 @@ latex_documents = [
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'djangoq2', 'Django Q2 Documentation',
[author], 1)
]
man_pages = [(master_doc, "djangoq2", "Django Q2 Documentation", [author], 1)]
# If true, show URL addresses after external links.
@@ -296,9 +291,15 @@ man_pages = [
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'DjangoQ2', 'Django Q2 Documentation',
author, 'DjangoQ2', 'A multiprocessing distributed task queue for Django.',
'Miscellaneous'),
(
master_doc,
"DjangoQ2",
"Django Q2 Documentation",
author,
"DjangoQ2",
"A multiprocessing distributed task queue for Django.",
"Miscellaneous",
),
]
# Documents to append as an appendix to all manuals.

View File

@@ -20,7 +20,18 @@ Configuration is handled via the ``Q_CLUSTER`` dictionary in your :file:`setting
'redis': {
'host': '127.0.0.1',
'port': 6379,
'db': 0, }
'db': 0, },
'ALT_CLUSTERS': {
'long': {
'timeout': 3000,
'retry': 3600,
'max_attempts': 2,
},
'short': {
'timeout': 10,
'max_attempts': 1,
},
}
}
All configuration settings are optional:
@@ -68,8 +79,17 @@ timeout
The number of seconds a worker is allowed to spend on a task before it's terminated. Defaults to ``None``, meaning it will never time out.
Set this to something that makes sense for your project. Can be overridden for individual tasks.
Note: for systems that don't have `SIGALRM` available (e.g. Windows), it will not raise an error properly. It will kill the task, but it will keep retrying until it finishes within the given time.
See :ref:`retry` for details how to set values for timeout and retry.
.. _time_zone:
time_zone
~~~~~~~
The timezone that is used for task scheduling. Use this if you are having issue with DST. The scheduler uses UTC to calculate the next date and will therefore ignore any DST changes. This will cause 1 hour or 0.5 hour changes in the schedule when time is moved one hour ahead or back. Defaults to `settings.TIME_ZONE` if `USE_TZ` is enabled.
.. _ack_failures:
ack_failures
@@ -310,16 +330,6 @@ Using the Django ORM backend will also enable the Queued Tasks table in the Admi
If you need better performance , you should consider using a different database backend than the main project.
Set ``orm`` to the name of that database connection and make sure you run migrations on it using the ``--database`` option.
When using the Django database as a message broker, you can set the ``has_replica`` boolean keyword to ensure Django-Q2 works properly letting a `Database Router <https://docs.djangoproject.com/en/3.2/topics/db/multi-db/>`__. ::
# example ORM broker connection with replica database
Q_CLUSTER = {
...
'orm': 'default',
'has_replica': True
}
.. _mongo_configuration:
mongo
@@ -340,7 +350,7 @@ To use MongoDB as a message broker you simply provide the connection information
}
}
The ``mongo`` dictionary can contain any of the parameters exposed by pymongo's `MongoClient <https://api.mongodb.org/python/current/api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient>`__
The ``mongo`` dictionary can contain any of the parameters exposed by pymongo's `MongoClient <https://pymongo.readthedocs.io/en/stable/api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient>`__
If you want to use a mongodb uri, you can supply it as the ``host`` parameter.
mongo_db
@@ -462,6 +472,24 @@ As a rule of thumb; cpu_affinity 1 favors repetitive short running tasks, while
*Psutil does not support cpu affinity on OS X at this time.*
.. _alt-clusters:
ALT_CLUSTERS
~~~~~~~~~~~~
For multiple clusters working on multiple queues to run in one Django site.
ALT_CLUSTERS should be a dict with cluster_name as its key, and the value is the configuration for the cluster
with the key as its name. The configuration items are consistent with Q_CLUSTER,
except for a few items such as name/cluster_name/ALT_CLUSTER, which are not available of course.
See :ref:`multiple-queues`.
.. note::
For a cluster, if its name is in ALT_CLUSTERS, the config item in ALT_CLUSTER will override
the same config item in the Q_CLUSTER root. Other config items in Q_CLUSTER root remain in effect for this cluster.
.. py:module:: django_q
.. rubric:: Footnotes

View File

@@ -1,4 +1,3 @@
version: "3.0"
services:
docs:
container_name: djangoq2-docs

View File

@@ -328,7 +328,7 @@ Requires cache to be enabled. Save file in your Django project's root directory
# All django stuff has to come after the setup:
django.setup()
from django_q.monitor import Stat
from django_q.status import Stat
from django_q.conf import Conf
# Set host and port settings

View File

@@ -27,7 +27,7 @@ Features
- Rollbar and Sentry support
Django Q2 is tested with: Python 3.8, 3.9 and 3.10, 3.11, Django 3.2.x and 4.1.x
Django Q2 is tested with: Python 3.8, 3.9, 3.10, 3.11 and 3.12. Works with Django 4.2.x, 5.x and 6.0.x
Currently available in English, German and French.

View File

@@ -24,15 +24,31 @@ Installation
$ python manage.py qcluster
Migrate from Django-Q to Django-Q2
----------------------------------
If you have an application with django-q running right now, you can simply swap the libraries and you should be good to go.::
$ pip uninstall django-q # you might have to uninstall django-q add-ons as well
$ pip install django-q2
Then migrate the database to get the latest tables/fields::
$ python manage.py migrate
Requirements
------------
Django Q2 is tested for Python 3.8, 3.9, 3.10 and 3.11
Django Q2 is tested for Python 3.8, 3.9, 3.10, 3.11 and 3.12
- `Django <https://www.djangoproject.com>`__
Django Q2 aims to use as much of Django's standard offerings as possible.
The code is tested against Django versions `3.2.x` and `4.1.x`.
The code is tested against Django versions `3.2.x`, `4.1.x`, `4.2.x` and `5.0.x`.
- `Django-picklefield <https://github.com/gintas/django-picklefield>`__
@@ -55,6 +71,10 @@ Optional
$ pip install psutil
- `setproctitle <https://github.com/dvarrazzo/py-setproctitle>`__ python module to customize the process title by Daniele Varrazzo', is an optional requirement used to set informative process titles::
$ pip install setproctitle
- `Hiredis <https://github.com/redis/hiredis>`__ parser. This C library maintained by the core Redis team is faster than the standard PythonParser during high loads::
$ pip install hiredis
@@ -95,11 +115,11 @@ Add-ons
-------
- `django-q-rollbar <https://github.com/danielwelch/django-q-rollbar>`__ is a Rollbar error reporter::
$ pip install django-q[rollbar]
$ pip install django-q2[rollbar]
- `django-q-sentry <https://github.com/danielwelch/django-q-sentry>`__ is a Sentry error reporter::
$ pip install django-q[sentry]
$ pip install django-q2[sentry]
- `django-q-email <https://github.com/joeyespo/django-q-email>`__ is a compatible Django email backend that will automatically async queue your emails.
@@ -125,7 +145,7 @@ Other known issues are:
Python
~~~~~~
Current tests are performed with 3.8, 3.9, 3.10 and 3.11
Current tests are performed with 3.8, 3.9, 3.10, 3.11 and 3.12
If you do encounter any regressions with earlier versions, please submit an issue on `github <https://github.com/GDay/django-q2>`__
Open-source packages
@@ -135,8 +155,8 @@ You can reference the `requirements <https://github.com/GDay/django-q2/blob/mast
Django
~~~~~~
We strive to be compatible with last two major version of Django.
At the moment this means we support the 3.2.x and 4.1.x releases.
We strive to be compatible with the last two major version of Django.
At the moment this means we support the 3.2.x, 4.1.x, 4.2.x and 5.0.x releases.
Since we are now no longer supporting Python 2, we can also not support older versions of Django that do not support Python >= 3.6
Since we are now no longer supporting Python 2, we can also not support older versions of Django that do not support Python >= 3.8
For this you can always use older releases, but they are no longer maintained.

View File

@@ -111,7 +111,7 @@ You can check the status of your clusters straight from your code with the :clas
.. code:: python
from django_q.monitor import Stat
from django_q.status import Stat
for stat in Stat.get_all():
print(stat.cluster_id, stat.status)

View File

@@ -69,6 +69,10 @@ You can change this by setting the :ref:`catch_up` configuration setting to ``Fa
The scheduler will then skip execution of scheduled events in the past.
Instead those tasks will run once when the cluster starts again and the scheduler will find the next available slot in the future according to original schedule parameters.
When :ref:`catch_up` is to ``True`` it may be useful for the task to know what was the date and time it was originally intended to run at.
To achieve this, pass an identifier name to parameter `intended_date_kwarg` when creating the schedule. The intended datetime will then be passed - in isoformat string - as
a kwarg with that identifier name to the task that has been created.
Management Commands
-------------------
@@ -123,9 +127,10 @@ Reference
:param int repeats: Number of times to repeat schedule. -1=Always, 0=Never, n =n.
:param datetime next_run: Next or first scheduled execution datetime.
:param str cluster: optional cluster name. Task will be executed only on a cluster with a matching :ref:`name`.
:param str intended_date_kwarg: optional identifier to pass intended schedule date.
:param dict q_options: options passed to async_task for this schedule
:param kwargs: optional keyword arguments for the scheduled function.
.. note::
q_options does not accept the 'broker' key with a broker instance but accepts a 'broker_name' key instead. This can be used to specify the broker connection name to assign the task. If a broker with the specified name does not exist or is not running at the moment of placing the task in queue it fallbacks to the random broker/queue that handled the schedule.
@@ -183,9 +188,13 @@ Reference
When set to -1, this will keep counting down.
.. py:attribute:: cluster
Task will be executed only on a cluster with a matching :ref:`name`.
.. py:attribute:: intended_date_kwarg
Name of kwarg to pass intended schedule date.
.. py:attribute:: next_run
Datetime of the next scheduled execution.

View File

@@ -13,6 +13,12 @@ Before enqueuing a task
The ``django_q.signals.pre_enqueue`` signal is emitted before a task is
enqueued. The task dictionary is given as the ``task`` argument.
After spawning a worker process
"""""""""""""""""""""""""""""""
The ``django_q.signals.post_spawn`` signal is emitted after a worker process has
spawned. The process name is given as the ``proc_name`` argument (string).
Before executing a task
"""""""""""""""""""""""
@@ -26,8 +32,14 @@ executed by a worker. This signal provides two arguments:
After executing a task
""""""""""""""""""""""
The ``django_q.signals.post_execute`` signal is emitted after a task is
executed by a worker and processed by the monitor. It included the ``task`` dictionary with the result.
- The ``django_q.signals.post_execute_in_worker`` signal is emitted after a task
is executed by a worker and processed by the **worker**. It included the ``task``
dictionary with the result. Note that this signal is **emitted from, and handled
by, the worker process itself**, not the monitor, unlike the ``post_execute``
signal below.
- The ``django_q.signals.post_execute`` signal is emitted after a task is
executed by a worker and processed by the **monitor**. It included the ``task``
dictionary with the result.
Subscribing to a signal
@@ -37,7 +49,7 @@ Connecting to a Django Q2 signal is done the same as any other Django
signal::
from django.dispatch import receiver
from django_q.signals import pre_enqueue, pre_execute, post_execute
from django_q.signals import pre_enqueue, pre_execute, post_execute, post_spawn
@receiver(pre_enqueue)
def my_pre_enqueue_callback(sender, task, **kwargs):
@@ -50,5 +62,13 @@ signal::
@receiver(post_execute)
def my_post_execute_callback(sender, task, **kwargs):
print(f"Task {task['name']} was executed with result {task['result']}")
@receiver(post_execute_in_worker)
def my_post_execute_in_worker_callback(sender, func, task, **kwargs):
print(f"Task {task['name']} was executed with result {task['result']}")
@receiver(post_spawn)
def my_post_spawn_callback(sender, proc_name, **kwargs):
print(f"Process {proc_name} has spawned")

View File

@@ -75,6 +75,10 @@ broker
""""""
A broker instance, in case you want to control your own connections.
cluster
""""""
The name of the cluster. Only useful if you are using [alternative queues](https://django-q2.readthedocs.io/en/master/cluster.html#multiple-queues).
task_name
"""""""""
@@ -121,6 +125,7 @@ Optionally you can use the :class:`AsyncTask` class to instantiate a task and ke
a.run()
# wait indefinitely for the result and print it
# don't let the task return `None` or it will wait indefinitely
print(a.result(wait=-1))
# change the args
@@ -240,7 +245,7 @@ Reference
---------
.. py:function:: async_task(func, *args, hook=None, group=None, timeout=None,\
save=None, sync=False, cached=False, broker=None, q_options=None, **kwargs)
save=None, sync=False, cached=False, broker=None, cluster=None, q_options=None, **kwargs)
Puts a task in the cluster queue
@@ -254,6 +259,7 @@ Reference
:param bool sync: If set to True, async_task will simulate a task execution
:param cached: Output the result to the cache backend. Bool or timeout in seconds
:param broker: Optional broker connection from :func:`brokers.get_broker`
:param cluster: Optional cluster name if using alternative queues
:param dict q_options: Options dict, overrides option keywords
:param dict kwargs: Keyword arguments for the task function
:returns: The uuid of the task
@@ -264,7 +270,7 @@ Reference
Gets the result of a previously executed task
:param str task_id: the uuid or name of the task
:param int wait: optional milliseconds to wait for a result. -1 for indefinite
:param int wait: optional milliseconds to wait for a result. -1 for indefinite, but be sure the result will not be `None` otherwise it will wait indefinitely!
:param bool cached: run this against the cache backend.
:returns: The result of the executed task

View File

16
exampleproject/asgi.py Normal file
View File

@@ -0,0 +1,16 @@
"""
ASGI config for exampleproject project.
It exposes the ASGI callable as a module-level variable named ``application``.
For more information on this file, see
https://docs.djangoproject.com/en/5.1/howto/deployment/asgi/
"""
import os
from django.core.asgi import get_asgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "exampleproject.settings")
application = get_asgi_application()

134
exampleproject/settings.py Normal file
View File

@@ -0,0 +1,134 @@
"""
Django settings for exampleproject project.
Generated by 'django-admin startproject' using Django 5.1.1.
For more information on this file, see
https://docs.djangoproject.com/en/5.1/topics/settings/
For the full list of settings and their values, see
https://docs.djangoproject.com/en/5.1/ref/settings/
"""
from pathlib import Path
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/5.1/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = "django-insecure-)oouh93bjg+c=b!l5-*w)7et1l+!3nmp223rr^1r4v#jn&ow3f"
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
ALLOWED_HOSTS = ["*"]
# Application definition
INSTALLED_APPS = [
"django.contrib.admin",
"django.contrib.auth",
"django.contrib.contenttypes",
"django.contrib.sessions",
"django.contrib.messages",
"django.contrib.staticfiles",
"django_q",
]
MIDDLEWARE = [
"django.middleware.security.SecurityMiddleware",
"django.contrib.sessions.middleware.SessionMiddleware",
"django.middleware.common.CommonMiddleware",
"django.middleware.csrf.CsrfViewMiddleware",
"django.contrib.auth.middleware.AuthenticationMiddleware",
"django.contrib.messages.middleware.MessageMiddleware",
"django.middleware.clickjacking.XFrameOptionsMiddleware",
]
ROOT_URLCONF = "exampleproject.urls"
TEMPLATES = [
{
"BACKEND": "django.template.backends.django.DjangoTemplates",
"DIRS": [],
"APP_DIRS": True,
"OPTIONS": {
"context_processors": [
"django.template.context_processors.debug",
"django.template.context_processors.request",
"django.contrib.auth.context_processors.auth",
"django.contrib.messages.context_processors.messages",
],
},
},
]
WSGI_APPLICATION = "exampleproject.wsgi.application"
# Database
# https://docs.djangoproject.com/en/5.1/ref/settings/#databases
DATABASES = {
"default": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": BASE_DIR / "db.sqlite3",
}
}
# Password validation
# https://docs.djangoproject.com/en/5.1/ref/settings/#auth-password-validators
AUTH_PASSWORD_VALIDATORS = [
{
"NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator",
},
{
"NAME": "django.contrib.auth.password_validation.MinimumLengthValidator",
},
{
"NAME": "django.contrib.auth.password_validation.CommonPasswordValidator",
},
{
"NAME": "django.contrib.auth.password_validation.NumericPasswordValidator",
},
]
# Internationalization
# https://docs.djangoproject.com/en/5.1/topics/i18n/
LANGUAGE_CODE = "en-us"
TIME_ZONE = "UTC"
USE_I18N = True
USE_TZ = True
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/5.1/howto/static-files/
STATIC_URL = "static/"
# Default primary key field type
# https://docs.djangoproject.com/en/5.1/ref/settings/#default-auto-field
DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
Q_CLUSTER = {
"name": "DjangORM",
"workers": 4,
"timeout": 90,
"retry": 120,
"queue_limit": 50,
"bulk": 10,
"orm": "default",
}

27
exampleproject/urls.py Normal file
View File

@@ -0,0 +1,27 @@
"""
URL configuration for exampleproject project.
The `urlpatterns` list routes URLs to views. For more information please see:
https://docs.djangoproject.com/en/5.1/topics/http/urls/
Examples:
Function views
1. Add an import: from my_app import views
2. Add a URL to urlpatterns: path('', views.home, name='home')
Class-based views
1. Add an import: from other_app.views import Home
2. Add a URL to urlpatterns: path('', Home.as_view(), name='home')
Including another URLconf
1. Import the include() function: from django.urls import include, path
2. Add a URL to urlpatterns: path('blog/', include('blog.urls'))
"""
from django.contrib import admin
from django.urls import path
from exampleproject import views
urlpatterns = [
path("admin/", admin.site.urls),
path("new_task/", views.add_task, name="add_task"),
path("result/<slug:task_id>/", views.get_result, name="get_result"),
]

31
exampleproject/views.py Normal file
View File

@@ -0,0 +1,31 @@
import time
from django.http import HttpResponse
from django.urls import reverse
from django_q import tasks
# internal function to be called with django_q
def new_task(run_for_minutes):
print("Task started")
time.sleep(run_for_minutes)
print("Task done")
return True
def add_task(request):
task_id = tasks.async_task(new_task, 5)
result_url = reverse("get_result", args=[task_id])
return HttpResponse(
f"Added async task with <a href='{result_url}'>Go to results</a>"
)
def get_result(request, task_id):
task = tasks.fetch(task_id)
if not task:
msg = "Task running... please refresh after some time"
else:
msg = f"Async task result: {task.result}"
return HttpResponse(msg)

16
exampleproject/wsgi.py Normal file
View File

@@ -0,0 +1,16 @@
"""
WSGI config for exampleproject project.
It exposes the WSGI callable as a module-level variable named ``application``.
For more information on this file, see
https://docs.djangoproject.com/en/5.1/howto/deployment/wsgi/
"""
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "exampleproject.settings")
application = get_wsgi_application()

1960
poetry.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,10 @@
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
[tool.poetry]
name = "django-q2"
version = "1.4.5"
version = "1.10.0"
packages = [
{ include = "django_q" },
]
@@ -26,15 +30,19 @@ classifiers = [
'Operating System :: MacOS',
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
'Programming Language :: Python :: 3.12',
'Programming Language :: Python :: 3.13',
'Topic :: Internet :: WWW/HTTP',
'Topic :: System :: Distributed Computing',
'Topic :: Software Development :: Libraries :: Python Modules',
]
include = ['CHANGELOG.md']
include = [
{ path = "CHANGELOG.md", format = "sdist" },
]
[tool.poetry.plugins] # Optional super table
@@ -44,8 +52,8 @@ include = ['CHANGELOG.md']
[tool.poetry.dependencies]
python = ">=3.8.14, <4"
django = ">=3.2"
python = ">=3.9,<4"
django = ">=4.2"
django-picklefield = "^3.1"
blessed = { version = "^1.19.1", optional = true }
@@ -55,10 +63,12 @@ django-redis = { version = "^5.2.0", optional = true }
iron-mq = { version = "^0.9", optional = true }
boto3 = { version = "^1.24.92", optional = true }
pymongo = { version = "^4.2.0", optional = true }
croniter = { version = "^1.3.7", optional = true }
croniter = { version = "^2.0.1", optional = true }
django-q-rollbar = {version = ">=0.1", optional = true}
django-q-sentry = {version = ">=0.1", optional = true}
redis = {version = "^4.3.4", optional = true}
setproctitle = {version = "^1.3.2", optional = true}
importlib-metadata = {version = ">=3.6", python = "<3.10"}
[tool.poetry.dev-dependencies]
@@ -66,13 +76,12 @@ pytest = "^7.1.3"
pytest-django = "^4.5.2"
Sphinx = "^4.0.2"
pytest-cov = "^4.0.0"
black = "^22.10.0"
isort = {extras = ["requirements_deprecated_finder"], version = "^5.10.1"}
ruff = "^0.4.4"
[tool.poetry.extras]
requires = ["poetry_core>=1.0.0"]
requires = ["poetry_core"]
build-backend = ["poetry.core.masonry.api"]
testing = ["django-redis", "croniter", "hiredis", "psutil", "iron-mq", "boto3", "pymongo", "blessed", "redis"]
testing = ["django-redis", "croniter", "hiredis", "psutil", "iron-mq", "boto3", "pymongo", "blessed", "redis", "setproctitle"]
rollbar = ["django-q-rollbar"]
sentry = ["django-q-sentry"]

View File

@@ -1,40 +1,41 @@
ansicon==1.89.0; platform_system == "Windows" and python_version >= "2.7" \
--hash=sha256:f1def52d17f65c2c9682cf8370c03f541f410c1752d6a14029f97318e4b9dfec \
--hash=sha256:e4d039def5768a47e4afec8e89e83ec3ae5a26bf00ad851f914d1240b444d2b1
arrow==1.1.1; python_version >= "3.6" \
--hash=sha256:77a60a4db5766d900a2085ce9074c5c7b8e2c99afeaa98ad627637ff6f292510 \
--hash=sha256:dee7602f6c60e3ec510095b5e301441bc56288cb8f51def14dcb3079f623823a
asgiref==3.3.4; python_version >= "3.6" \
--hash=sha256:92906c611ce6c967347bbfea733f13d6313901d54dcca88195eaeb52b2a8e8ee \
--hash=sha256:d1216dfbdfb63826470995d31caed36225dcaf34f182e0fa257a4dd9e86f1b78
blessed==1.18.1; python_version >= "2.7" \
--hash=sha256:dd7c0d33db9a2e7f597b446996484d0ed46e1586239db064fb5025008937dcae \
--hash=sha256:8b09936def6bc06583db99b65636b980075733e13550cb6af262ce724a55da23
django-picklefield==3.0.1; python_version >= "3" \
--hash=sha256:15ccba592ca953b9edf9532e64640329cd47b136b7f8f10f2939caa5f9ce4287 \
--hash=sha256:3c702a54fde2d322fe5b2f39b8f78d9f655b8f77944ab26f703be6c0ed335a35
django==3.2.4; python_version >= "3.6" \
--hash=sha256:ea735cbbbb3b2fba6d4da4784a0043d84c67c92f1fdf15ad6db69900e792c10f \
--hash=sha256:66c9d8db8cc6fe938a28b7887c1596e42d522e27618562517cc8929eb7e7f296
jinxed==1.1.0; platform_system == "Windows" and python_version >= "2.7" \
--hash=sha256:6a61ccf963c16aa885304f27e6e5693783676897cea0c7f223270c8b8e78baf8 \
--hash=sha256:d8f1731f134e9e6b04d95095845ae6c10eb15cb223a5f0cabdea87d4a279c305
python-dateutil==2.8.1; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version >= "3.6" \
--hash=sha256:73ebfe9dbf22e832286dafa60473e4cd239f8592f699aa5adaf10050e6e1823c \
--hash=sha256:75bb3f31ea686f1197762692a9ee6a7550b59fc6ca3a1f4b5d7e32fb98e2da2a
pytz==2021.1; python_version >= "3.6" \
--hash=sha256:eb10ce3e7736052ed3623d49975ce333bcd712c7bb19a58b9e2089d4057d0798 \
--hash=sha256:83a4a90894bf38e243cf052c8b58f381bfe9a7a483f6a9cab140bc7f702ac4da
six==1.16.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version >= "3.6" \
--hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254 \
--hash=sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926
sqlparse==0.4.1; python_version >= "3.6" \
--hash=sha256:017cde379adbd6a1f15a61873f43e8274179378e95ef3fede90b5aa64d304ed0 \
--hash=sha256:0f91fd2e829c44362cbcfab3e9ae12e22badaa8a29ad5ff599f9ec109f0454e8
typing-extensions==3.10.0.0; python_version < "3.8" and python_version >= "3.6" \
--hash=sha256:0ac0f89795dd19de6b97debb0c6af1c70987fd80a2d62d1958f7e56fcc31b497 \
--hash=sha256:779383f6086d90c99ae41cf0ff39aac8a7937a9283ce0a414e5dd782f4c94a84 \
--hash=sha256:50b6f157849174217d0656f99dc82fe932884fb250826c18350e159ec6cdf342
wcwidth==0.2.5; python_version >= "2.7" \
--hash=sha256:beb4802a9cebb9144e99086eff703a642a13d6a0052920003a230f3294bbe784 \
--hash=sha256:c4d647b99872929fdb7bdcaa4fbe7f01413ed3d98077df798530e5b04f116c83
asgiref==3.8.1 ; python_version >= "3.8" and python_version < "4" \
--hash=sha256:3e1e3ecc849832fe52ccf2cb6686b7a55f82bb1d6aee72a58826471390335e47 \
--hash=sha256:c343bd80a0bec947a9860adb4c432ffa7db769836c64238fc34bdc3fec84d590
backports-zoneinfo==0.2.1 ; python_version >= "3.8" and python_version < "3.9" \
--hash=sha256:17746bd546106fa389c51dbea67c8b7c8f0d14b5526a579ca6ccf5ed72c526cf \
--hash=sha256:1b13e654a55cd45672cb54ed12148cd33628f672548f373963b0bff67b217328 \
--hash=sha256:1c5742112073a563c81f786e77514969acb58649bcdf6cdf0b4ed31a348d4546 \
--hash=sha256:4a0f800587060bf8880f954dbef70de6c11bbe59c673c3d818921f042f9954a6 \
--hash=sha256:5c144945a7752ca544b4b78c8c41544cdfaf9786f25fe5ffb10e838e19a27570 \
--hash=sha256:7b0a64cda4145548fed9efc10322770f929b944ce5cee6c0dfe0c87bf4c0c8c9 \
--hash=sha256:8439c030a11780786a2002261569bdf362264f605dfa4d65090b64b05c9f79a7 \
--hash=sha256:8961c0f32cd0336fb8e8ead11a1f8cd99ec07145ec2931122faaac1c8f7fd987 \
--hash=sha256:89a48c0d158a3cc3f654da4c2de1ceba85263fafb861b98b59040a5086259722 \
--hash=sha256:a76b38c52400b762e48131494ba26be363491ac4f9a04c1b7e92483d169f6582 \
--hash=sha256:da6013fd84a690242c310d77ddb8441a559e9cb3d3d59ebac9aca1a57b2e18bc \
--hash=sha256:e55b384612d93be96506932a786bbcde5a2db7a9e6a4bb4bffe8b733f5b9036b \
--hash=sha256:e81b76cace8eda1fca50e345242ba977f9be6ae3945af8d46326d776b4cf78d1 \
--hash=sha256:e8236383a20872c0cdf5a62b554b27538db7fa1bbec52429d8d106effbaeca08 \
--hash=sha256:f04e857b59d9d1ccc39ce2da1021d196e47234873820cbeaad210724b1ee28ac \
--hash=sha256:fadbfe37f74051d024037f223b8e001611eac868b5c5b06144ef4d8b799862f2
django-picklefield==3.2 ; python_version >= "3.8" and python_version < "4" \
--hash=sha256:aa463f5d79d497dbe789f14b45180f00a51d0d670067d0729f352a3941cdfa4d \
--hash=sha256:e9a73539d110f69825d9320db18bcb82e5189ff48dbed41821c026a20497764c
django==4.2.11 ; python_version >= "3.8" and python_version < "4" \
--hash=sha256:6e6ff3db2d8dd0c986b4eec8554c8e4f919b5c1ff62a5b4390c17aff2ed6e5c4 \
--hash=sha256:ddc24a0a8280a0430baa37aff11f28574720af05888c62b7cfe71d219f4599d3
importlib-metadata==7.1.0 ; python_version >= "3.8" and python_version < "3.10" \
--hash=sha256:30962b96c0c223483ed6cc7280e7f0199feb01a0e40cfae4d4450fc6fab1f570 \
--hash=sha256:b78938b926ee8d5f020fc4772d487045805a55ddbad2ecf21c6d60938dc7fcd2
sqlparse==0.5.0 ; python_version >= "3.8" and python_version < "4" \
--hash=sha256:714d0a4932c059d16189f58ef5411ec2287a4360f17cdd0edd2d09d4c5087c93 \
--hash=sha256:c204494cd97479d0e39f28c93d46c0b2d5959c7b9ab904762ea6c7af211c8663
typing-extensions==4.11.0 ; python_version >= "3.8" and python_version < "3.11" \
--hash=sha256:83f085bd5ca59c80295fc2a82ab5dac679cbe02b9f33f7d83af68e241bea51b0 \
--hash=sha256:c1f94d72897edaf4ce775bb7558d5b79d8126906a14ea5ed1635921406c0387a
tzdata==2024.1 ; python_version >= "3.8" and python_version < "4" and sys_platform == "win32" \
--hash=sha256:2674120f8d891909751c38abcdfd386ac0a5a1127954fbc332af6b5ceae07efd \
--hash=sha256:9068bc196136463f5245e51efda838afa15aaeca9903f49050dfa2679db4d252
zipp==3.18.1 ; python_version >= "3.8" and python_version < "3.10" \
--hash=sha256:206f5a15f2af3dbaee80769fb7dc6f249695e940acca08dfb2a4769fe61e538b \
--hash=sha256:2884ed22e7d8961de1c9a05142eb69a247f120291bc0206a00a7642f09b5b715

1
ruff.toml Normal file
View File

@@ -0,0 +1 @@
lint.extend-select = ["I"]

View File

@@ -1,5 +1,3 @@
version: '3'
services:
redis:
image: redis:latest
@@ -15,15 +13,41 @@ services:
networks:
- main
aws:
container_name: aws
image: localstack/localstack:3.4.0
ports:
- "127.0.0.1:4566:4566" # LocalStack Gateway
- "127.0.0.1:4510-4559:4510-4559" # External services port range
environment:
AWS_DEFAULT_REGION: ${AWS_DEFAULT_REGION:-us-west-2}
DEFAULT_REGION: ${AWS_DEFAULT_REGION:-us-west-2}
SQS_ENDPOINT_STRATEGY: path
SERVICES: sqs
LOCALSTACK_HOST: aws
DEBUG: 1
LS_LOG: trace
volumes:
- ./containers/localstack:/etc/localstack/init/ready.d
networks:
- main
django-q2:
build:
dockerfile: ./Dockerfile.dev
context: .
environment:
AWS_ENDPOINT_URL: http://aws:4566
AWS_REGION: ${AWS_REGION:-us-west-2}
AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID:-test}
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY:-test}
AWS_DEFAULT_REGION: ${AWS_DEFAULT_REGION:-us-west-2}
volumes:
- .:/app
depends_on:
- redis
- mongo
- aws
networks:
- main

2
tox.ini Normal file
View File

@@ -0,0 +1,2 @@
[flake8]
max-line-length=88

Some files were not shown because too many files have changed in this diff Show More