42 Commits

Author SHA1 Message Date
GDay
f8501bfb4c Release v1.5.4 2023-06-29 01:03:57 +02:00
Scott Pashley
23c06a38eb Rerun successful tasks (#99)
* Move logic that resubmits a task and then deletes the original into an action function that can be used for both failure and success resubmission.

* Use the resubmit_task action in the FailAdmin.

* Add the resubmit_task action to the TaskAdmin class.

* Update resubmit failure test to reflect using the new resubmit action.

* Add a test for resubmitting successful tasks.

---------

Co-authored-by: Scott Pashley
2023-06-23 23:05:04 +02:00
GDay
f8321e7f54 Release v1.5.3 2023-05-14 03:22:34 +02:00
Stan Triepels
630b3c62b3 Make processes identifiable with uuid4 (#91)
* make processes identifiable

* update docs
2023-05-14 03:07:38 +02:00
Stan Triepels
227caa7446 Post spawn docs (#95) 2023-05-14 03:04:07 +02:00
David Hoover
4ba9e0338a Add post_spawn signal. (#93)
Co-authored-by: David Hoover <hooverd@arizona.edu>
2023-05-14 02:51:48 +02:00
GDay
fb960104c3 Release v1.5.2 2023-04-13 01:31:59 +02:00
GDay
88bbfd8967 Mention support for 4.2.x in docs 2023-04-13 01:21:46 +02:00
Martijn Jacobs
1f31725f43 Added Django 4.2 to the test matrix, fixed deprecation warning (#89)
* Added Django 4.2 to the CI test matrix

* Positional arguments with TimestampSigner are deprecated
2023-04-12 18:18:03 +02:00
GDay
322a53f06d Release v1.5.1 2023-04-02 17:53:23 +02:00
GDay
77821b1de7 Fix release workflow after name change 2023-04-02 17:51:17 +02:00
GDay
38d48231d6 Release v1.5.0 2023-04-02 17:43:17 +02:00
Stan Triepels
d13e9d3d64 Update dependencies (2023-04-02) (#88) 2023-04-02 17:15:39 +02:00
sinowood
52c04217e9 Multiple queue, multiple cluster in one site (#71)
* Support for multi-queue, multi-cluster configuration. API changes include:
  * Adding `cluster` to async_task() parameters and Task model
  * Adding argument --name to qcluster command
  * Necessary adjustments to Conf and Broker classes
  * Some admin improvements

* Add settings.Q_CLUSTER['ALT_CLUSTERS']: q_cluster config overrides for alternative clusters;
Add Conf.CLUSTER_NAME: separate usage from Conf.PREFIX;
QueueAdmin/OrmQ detail page enhanced: now displaying args/kwargs/q_options instead of encrypted payload.

* if `cluster` argument is not set (the default), async_task() and schedule() will be handled by the default cluster; Documentation update.

* Text cleanup

* Documentation update.

* Fix TIMEOUT setting in Windows for non-default cluster

---------

Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
2023-04-02 16:36:17 +02:00
Stan Triepels
f8520c9cda Remove version locking on poetry_core to fix regex error (#87) 2023-03-31 02:43:48 +02:00
GDay
86467dcf03 Revert "Merge pull request #15 from django-q2/error-on-timeout"
This reverts commit d97dca5bcb, reversing
changes made to 01cb652071.
2023-03-31 02:22:38 +02:00
Stan Triepels
69a40344f0 Allow building docs all formats (#77) 2023-02-23 01:20:19 +01:00
Jim Jagielski
d97dca5bcb Merge pull request #15 from django-q2/error-on-timeout
Error on timeout
2023-02-17 12:41:27 -05:00
Jim Jagielski
f5ecd337ad Merge branch 'master' into error-on-timeout 2023-02-17 12:33:47 -05:00
GDay
01cb652071 Release v1.4.11 2023-01-31 00:24:44 +01:00
Stan Triepels
b9bdd64568 Remove custom build (revert to auto create setup file) (#70) 2023-01-31 00:21:37 +01:00
Stan Triepels
b2bcffbcab Fix missing setup file for "No matching distribution" error (#69) 2023-01-31 00:09:33 +01:00
GDay
25cd180410 Release v1.4.10 2023-01-27 02:10:20 +01:00
Stan Triepels
f9e01699c5 Adding translation mo files automatically on build (#65) 2023-01-26 16:45:44 +01:00
Stan Triepels
2f3b0d0a31 Update all dependencies (#64) 2023-01-26 02:56:20 +01:00
Stan Triepels
264607f59c Bump translations to latest changes (#63) 2023-01-26 02:49:46 +01:00
msabatier
17c1609f10 Add meaningfull process titles with currently running task name (#57)
* Customize process names (in ps/top) and add task name in logs

* Increase severity of reincarnate log messages and add task name

* Document setproctitle optional dependency

Co-authored-by: Marc Sabatier <37879561+msabatier@users.noreply.github.com>
Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
2023-01-26 02:21:56 +01:00
msabatier
650c3b1524 Fix use of database router for write queries and remove Conf.HAS_REPLICA (#61)
Co-authored-by: Marc Sabatier
2023-01-18 02:12:13 +01:00
msabatier
a013591de5 Add intended_date_kwarg field to Schedule (#62)
Co-authored-by: Marc Sabatier <marc.sabatier@eco-adapt.com>
2023-01-17 22:52:11 +01:00
msabatier
865b1a5ba8 Change task timeout logic to have now() as execution time (#58) 2023-01-11 01:48:57 +01:00
msabatier
8cd1028391 More explicit log messages in exception handling (#59) 2023-01-11 00:36:35 +01:00
GDay
31e82ad028 Release v1.4.9 2022-12-22 03:06:01 +01:00
Stan Triepels
b66dfbd3af Fix DST timezone change (move from DST to normal jump) (#56) 2022-12-22 03:02:53 +01:00
GDay
08baaab125 Release v1.4.8 2022-12-21 03:46:01 +01:00
Stan Triepels
ce81059da8 Fix: allow both ZoneInfo and Pytz depending on django version (#55) 2022-12-21 03:42:38 +01:00
Stan Triepels
cf33275d92 Release v1.4.7 (#54) 2022-12-21 01:53:10 +01:00
Stan Triepels
e78e473be3 Fix: handling exceptions inside job function (#51) 2022-12-21 01:44:59 +01:00
Stan Triepels
4d454e4001 Fix: Daylight saving time issue with scheduler (#47) 2022-12-21 01:44:34 +01:00
Stan Triepels
cf33891291 Chore: Remove release drafter (#53) 2022-12-21 01:44:10 +01:00
Stan Triepels
134a54dbeb Chore: Fix badge and add download badge (#52) 2022-12-21 01:43:51 +01:00
Noortheen Raja
64d73a77b7 feat: option to fail-on-timeout
fixes #251
2022-10-19 02:27:12 +02:00
Noortheen Raja
596842e692 chore: pip install support
having build-system config helps installing  this without setup.py built
2022-10-19 02:27:10 +02:00
43 changed files with 2080 additions and 1476 deletions

View File

@@ -1,37 +0,0 @@
categories:
-
label: breaking
title: Breaking
-
label: feature
title: New
-
label: bug
title: "Bug Fixes"
-
label: dependencies
title: "Dependency Updates"
-
label: security
title: Security
name-template: v$NEXT_PATCH_VERSION
tag-template: v$NEXT_PATCH_VERSION
template: |
# Changes
$CHANGES
version-resolver:
major:
labels:
- breaking
- major
minor:
labels:
- feature
- minor
patch:
labels:
- bug
- dependencies
- security
- patch
default: patch

View File

@@ -7,7 +7,7 @@ on:
jobs:
build:
if: github.repository == 'GDay/django-q2'
if: github.repository == 'django-q2/django-q2'
runs-on: ubuntu-latest
steps:

View File

@@ -1,14 +0,0 @@
name: Update release draft
on:
push:
branches:
- master
jobs:
update_release_draft:
runs-on: ubuntu-latest
steps:
- uses: release-drafter/release-drafter@v5
with:
config-name: release-drafter.yml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -12,7 +12,7 @@ jobs:
strategy:
matrix:
python-version: [ "3.8", "3.9", "3.10", "3.11" ]
django: [ "3.2", "4.1" ]
django: [ "3.2", "4.1", "4.2" ]
services:
disque:

View File

@@ -15,6 +15,8 @@ build:
sphinx:
configuration: docs/conf.py
formats: all
python:
install:
- requirements: docs/requirements.txt

View File

@@ -2,6 +2,80 @@
## [Unreleased](https://github.com/GDay/django-q2/tree/HEAD)
## [v1.5.4](https://github.com/GDay/django-q2/tree/v1.5.4) (2023-06-29)
**Merged pull requests:**
- Rerun successful tasks https://github.com/django-q2/django-q2/pull/99
## [v1.5.3](https://github.com/GDay/django-q2/tree/v1.5.3) (2023-05-14)
**Merged pull requests:**
- Add post_spawn signal. https://github.com/django-q2/django-q2/pull/93
- Post spawn docs https://github.com/django-q2/django-q2/pull/95
- Make processes identifiable with uuid4 https://github.com/django-q2/django-q2/pull/91
## [v1.5.2](https://github.com/GDay/django-q2/tree/v1.5.2) (2023-04-13)
**Merged pull requests:**
- Added Django 4.2 to the test matrix, fixed deprecation warning https://github.com/GDay/django-q2/pull/89
- Updated docs to show support for 4.2
## [v1.5.1](https://github.com/GDay/django-q2/tree/v1.5.1) (2023-04-02)
- Fix release to pipy due to changed org name
## [v1.5.0](https://github.com/GDay/django-q2/tree/v1.5.0) (2023-04-02)
**Merged pull requests:**
- Multiple queue, multiple cluster in one site https://github.com/GDay/django-q2/pull/71
- Allow building docs all formats https://github.com/GDay/django-q2/pull/77
- Remove version locking on `poetry_core` to fix regex error https://github.com/GDay/django-q2/pull/87
- Update dependencies (2023-04-02) https://github.com/GDay/django-q2/pull/88
## [v1.4.11](https://github.com/GDay/django-q2/tree/v1.4.11) (2023-01-30)
**Merged pull requests:**
- Fix missing setup file for "No matching distribution" error https://github.com/GDay/django-q2/pull/69
- Remove custom build (revert to auto create setup file) https://github.com/GDay/django-q2/pull/70
## [v1.4.10](https://github.com/GDay/django-q2/tree/v1.4.10) (2023-01-26)
**Merged pull requests:**
- Adding translation mo files automatically on build https://github.com/GDay/django-q2/pull/65
- Update all dependencies https://github.com/GDay/django-q2/pull/64
- Bump translations to latest changes https://github.com/GDay/django-q2/pull/63
- Add meaningfull process titles with currently running task name https://github.com/GDay/django-q2/pull/57
- Fix use of database router for write queries and remove Conf.HAS_REPLICA https://github.com/GDay/django-q2/pull/61
- Add intended_date_kwarg field to Schedule https://github.com/GDay/django-q2/pull/62
- Change task timeout logic to have now() as execution time https://github.com/GDay/django-q2/pull/58
- More explicit log messages in exception handling https://github.com/GDay/django-q2/pull/59
## [v1.4.9](https://github.com/GDay/django-q2/tree/v1.4.9) (2022-12-22)
**Merged pull requests:**
- Fix DST timezone change (move from DST to normal jump) https://github.com/GDay/django-q2/pull/56
## [v1.4.8](https://github.com/GDay/django-q2/tree/v1.4.8) (2022-12-21)
**Merged pull requests:**
- Fix: allow both ZoneInfo and Pytz depending on django version https://github.com/GDay/django-q2/pull/55
## [v1.4.7](https://github.com/GDay/django-q2/tree/v1.4.7) (2022-12-21)
**Merged pull requests:**
- Fix: handling exceptions inside job function https://github.com/GDay/django-q2/pull/51
- Fix: Daylight saving time issue with scheduler https://github.com/GDay/django-q2/pull/47
- Chore: Fix badge and add download badge https://github.com/GDay/django-q2/pull/52
- Chore: Remove release drafter https://github.com/GDay/django-q2/pull/53
## [v1.4.6](https://github.com/GDay/django-q2/tree/v1.4.6) (2022-11-30)

View File

@@ -1,7 +1,7 @@
A multiprocessing distributed task queue for Django
---------------------------------------------------
|image0| |image1| |docs|
|image0| |image1| |docs| |downloads|
::
@@ -40,7 +40,7 @@ Requirements
- `Django <https://www.djangoproject.com>`__ > = 3.2
- `Django-picklefield <https://github.com/gintas/django-picklefield>`__
Tested with: Python 3.8, 3.9, 3.10, 3.11 Django 3.2.X and 4.1.X
Tested with: Python 3.8, 3.9, 3.10 and 3.11. Works with Django 3.2.X, 4.1.X and 4.2.X.
Brokers
~~~~~~~
@@ -245,4 +245,6 @@ Acknowledgements
.. |docs| image:: https://readthedocs.org/projects/docs/badge/?version=latest
:alt: Documentation Status
:scale: 100
:target: https://django-q.readthedocs.org/
:target: https://django-q2.readthedocs.org/
.. |downloads| image:: https://img.shields.io/pypi/dm/django-q2
:target: https://img.shields.io/pypi/dm/django-q2

View File

@@ -0,0 +1,9 @@
import subprocess
def generate_mo_files():
subprocess.run(["django-admin", "compilemessages"])
if __name__ == "__main__":
generate_mo_files()

View File

@@ -1,6 +1,6 @@
import django
VERSION = (1, 4, 6)
VERSION = (1, 5, 4)
if django.VERSION < (3, 2):
default_app_config = "django_q.apps.DjangoQConfig"

View File

@@ -10,10 +10,29 @@ from django_q.models import Failure, OrmQ, Schedule, Success, Task
from django_q.tasks import async_task
def resubmit_task(model_admin, request, queryset):
"""Submit selected tasks back to the queue."""
for task in queryset:
async_task(
task.func,
*task.args or (),
hook=task.hook,
group=task.group,
cluster=task.cluster,
**task.kwargs or {},
)
if isinstance(model_admin, FailAdmin):
task.delete()
resubmit_task.short_description = _("Resubmit selected tasks to queue")
class TaskAdmin(admin.ModelAdmin):
"""model admin for success tasks."""
list_display = ("name", "group", "func", "started", "stopped", "time_taken")
list_display = ("name", "group", "func", "cluster", "started", "stopped", "time_taken")
actions = [resubmit_task]
def has_add_permission(self, request):
"""Don't allow adds."""
@@ -26,35 +45,25 @@ class TaskAdmin(admin.ModelAdmin):
search_fields = ("name", "func", "group")
readonly_fields = []
list_filter = ("group",)
list_filter = ("group", "cluster")
def get_readonly_fields(self, request, obj=None):
"""Set all fields readonly."""
return list(self.readonly_fields) + [field.name for field in obj._meta.fields]
def retry_failed(FailAdmin, request, queryset):
"""Submit selected tasks back to the queue."""
for task in queryset:
async_task(task.func, *task.args or (), hook=task.hook, **task.kwargs or {})
task.delete()
retry_failed.short_description = _("Resubmit selected tasks to queue")
class FailAdmin(admin.ModelAdmin):
"""model admin for failed tasks."""
list_display = ("name", "group", "func", "started", "stopped", "short_result")
list_display = ("name", "group", "func", "cluster", "started", "stopped", "short_result")
def has_add_permission(self, request):
"""Don't allow adds."""
return False
actions = [retry_failed]
actions = [resubmit_task]
search_fields = ("name", "func", "group")
list_filter = ("group",)
list_filter = ("group", "cluster")
readonly_fields = []
def get_readonly_fields(self, request, obj=None):
@@ -123,6 +132,8 @@ class QueueAdmin(admin.ModelAdmin):
"""queue admin for ORM broker"""
list_display = ("id", "key", "name", "group", "func", "lock", "task_id")
fields = ("key", "lock", "task_id", "name", "group", "func", "args", "kwargs", "q_options")
readonly_fields = fields[2:]
def save_model(self, request, obj, form, change):
obj.save(using=Conf.ORM)

View File

@@ -7,7 +7,9 @@ from django_q.conf import Conf
class Broker:
def __init__(self, list_key: str = Conf.PREFIX):
def __init__(self, list_key: str = None):
# With same BROKER_CLASS, `list_key` is just a synonym for `queue_name` except for RedisBroker
list_key = list_key or Conf.CLUSTER_NAME
self.connection = self.get_connection(list_key)
self.list_key = list_key
self.cache = self.get_cache()
@@ -151,7 +153,7 @@ class Broker:
return None
@staticmethod
def get_connection(list_key: str = Conf.PREFIX):
def get_connection(list_key: str = None):
"""
Gets a connection to the broker
:param list_key: Optional queue name
@@ -160,13 +162,14 @@ class Broker:
return 0
def get_broker(list_key: str = Conf.PREFIX) -> Broker:
def get_broker(list_key: str = None) -> Broker:
"""
Gets the configured broker type
:param list_key: optional queue name
:type list_key: str
:return: a broker instance
"""
list_key = list_key or Conf.CLUSTER_NAME
# custom
if Conf.BROKER_CLASS:
module, func = Conf.BROKER_CLASS.rsplit(".", 1)

View File

@@ -8,7 +8,7 @@ QUEUE_DOES_NOT_EXIST = "AWS.SimpleQueueService.NonExistentQueue"
class Sqs(Broker):
def __init__(self, list_key: str = Conf.PREFIX):
def __init__(self, list_key: str = None):
self.sqs = None
super(Sqs, self).__init__(list_key)
self.queue = self.get_queue()
@@ -77,7 +77,7 @@ class Sqs(Broker):
return "AWS SQS"
@staticmethod
def get_connection(list_key: str = Conf.PREFIX) -> Session:
def get_connection(list_key: str = None) -> Session:
config = Conf.SQS
if "aws_region" in config:
config["region_name"] = config["aws_region"]

View File

@@ -46,6 +46,7 @@ class IronMQBroker(Broker):
return self.delete(task_id)
@staticmethod
def get_connection(list_key: str = Conf.PREFIX) -> Queue:
def get_connection(list_key: str = None) -> Queue:
list_key = list_key or Conf.CLUSTER_NAME
ironmq = IronMQ(name=None, **Conf.IRON_MQ)
return ironmq.queue(queue_name=list_key)

View File

@@ -15,7 +15,7 @@ def _timeout():
class Mongo(Broker):
def __init__(self, list_key=Conf.PREFIX):
def __init__(self, list_key: str = None):
super(Mongo, self).__init__(list_key)
self.collection = self.get_collection()
@@ -24,7 +24,7 @@ class Mongo(Broker):
self.collection = self.get_collection()
@staticmethod
def get_connection(list_key: str = Conf.PREFIX) -> MongoClient:
def get_connection(list_key: str = None) -> MongoClient:
return MongoClient(**Conf.MONGO)
def get_collection(self):

View File

@@ -11,12 +11,12 @@ from django_q.models import OrmQ
def _timeout():
return timezone.now() - timedelta(seconds=Conf.RETRY)
return timezone.now() + timedelta(seconds=Conf.RETRY)
class ORM(Broker):
@staticmethod
def get_connection(list_key: str = Conf.PREFIX):
def get_connection(list_key: str = None):
if transaction.get_autocommit(
using=Conf.ORM
): # Only True when not in an atomic block
@@ -31,13 +31,13 @@ class ORM(Broker):
def queue_size(self) -> int:
return (
self.get_connection()
.filter(key=self.list_key, lock__lte=_timeout())
.filter(key=self.list_key, lock__lte=timezone.now())
.count()
)
def lock_size(self) -> int:
return (
self.get_connection().filter(key=self.list_key, lock__gt=_timeout()).count()
self.get_connection().filter(key=self.list_key, lock__gt=timezone.now()).count()
)
def purge_queue(self):
@@ -55,13 +55,14 @@ class ORM(Broker):
self.delete(task_id)
def enqueue(self, task):
# list_key might be null (e.g. in a test setup) but OrmQ.key has not-null constraint
package = self.get_connection().create(
key=self.list_key, payload=task, lock=_timeout()
key=self.list_key or Conf.CLUSTER_NAME, payload=task, lock=timezone.now()
)
return package.pk
def dequeue(self):
tasks = self.get_connection().filter(key=self.list_key, lock__lt=_timeout())[
tasks = self.get_connection().filter(key=self.list_key, lock__lt=timezone.now())[
0 : Conf.BULK # noqa: E203
]
if tasks:
@@ -70,7 +71,7 @@ class ORM(Broker):
if (
self.get_connection()
.filter(id=task.id, lock=task.lock)
.update(lock=timezone.now())
.update(lock=_timeout())
):
task_list.append((task.pk, task.payload))
# else don't process, as another cluster has been faster than us on

View File

@@ -11,7 +11,8 @@ except ImportError:
class Redis(Broker):
def __init__(self, list_key: str = Conf.PREFIX):
def __init__(self, list_key: str = None):
list_key = list_key or Conf.CLUSTER_NAME
super(Redis, self).__init__(list_key=f"django_q:{list_key}:q")
def enqueue(self, task):
@@ -57,7 +58,7 @@ class Redis(Broker):
return self.connection.mget(keys)
@staticmethod
def get_connection(list_key: str = Conf.PREFIX) -> Redis:
def get_connection(list_key: str = None) -> Redis:
if django_redis and Conf.DJANGO_REDIS:
return django_redis.get_redis_connection(Conf.DJANGO_REDIS)
if isinstance(Conf.REDIS, str):

View File

@@ -20,7 +20,6 @@ except core.exceptions.AppRegistryNotReady:
django.setup()
from django.conf import settings
from django.utils import timezone
from django.utils.translation import gettext_lazy as _
@@ -34,37 +33,43 @@ from django_q.conf import (
get_ppid,
logger,
psutil,
setproctitle,
resource,
)
from django_q.humanhash import humanize
from django_q.models import Schedule, Success, Task
from django_q.queues import Queue
from django_q.signals import post_execute, pre_execute
from django_q.signals import post_execute, post_spawn, pre_execute
from django_q.signing import BadSignature, SignedPackage
from django_q.status import Stat, Status
from .utils import add_months, add_years, get_func_repr
from .utils import get_func_repr, localtime
class Cluster:
def __init__(self, broker: Broker = None):
self.broker = broker or get_broker()
# Cluster do not need an init or default broker except for testing,
# The sentinel will create a broker for cluster and utilize ALT_CLUSTERS config in Conf.
self.broker = broker # DON'T USE get_broker() to set a default broker here.
self.sentinel = None
self.stop_event = None
self.start_event = None
self.pid = current_process().pid
self.cluster_id = uuid.uuid4()
self.host = socket.gethostname()
self.timeout = Conf.TIMEOUT
self.timeout = None
signal.signal(signal.SIGTERM, self.sig_handler)
signal.signal(signal.SIGINT, self.sig_handler)
def start(self) -> int:
if setproctitle:
setproctitle.setproctitle(f"qcluster {current_process().name} {self.name}")
# Start Sentinel
self.stop_event = Event()
self.start_event = Event()
self.sentinel = Process(
target=Sentinel,
name=f"Process-{uuid.uuid4().hex}",
args=(
self.stop_event,
self.start_event,
@@ -139,7 +144,7 @@ class Sentinel:
start_event,
cluster_id,
broker=None,
timeout=Conf.TIMEOUT,
timeout=None,
start=True,
):
# Make sure we catch signals for the pool
@@ -156,7 +161,7 @@ class Sentinel:
self.start_event = start_event
self.pool_size = Conf.WORKERS
self.pool = []
self.timeout = timeout
self.timeout = timeout or Conf.TIMEOUT
self.task_queue = (
Queue(maxsize=Conf.QUEUE_LIMIT) if Conf.QUEUE_LIMIT else Queue()
)
@@ -167,6 +172,10 @@ class Sentinel:
if start:
self.start()
def queue_name(self):
# multi-queue: cluster name is (broker's) queue_name
return self.broker.list_key if self.broker else '--'
def start(self):
self.broker.ping()
self.spawn_cluster()
@@ -188,7 +197,7 @@ class Sentinel:
"""
:type target: function or class
"""
p = Process(target=target, args=args)
p = Process(target=target, args=args, name=f"Process-{uuid.uuid4().hex}")
p.daemon = True
if target == worker:
p.daemon = Conf.DAEMONIZE_WORKERS
@@ -218,13 +227,13 @@ class Sentinel:
db.connections.close_all()
if process == self.monitor:
self.monitor = self.spawn_monitor()
logger.error(
logger.critical(
_("reincarnated monitor %(name)s after sudden death")
% {"name": process.name}
)
elif process == self.pusher:
self.pusher = self.spawn_pusher()
logger.error(
logger.critical(
_("reincarnated pusher %(name)s after sudden death")
% {"name": process.name}
)
@@ -234,15 +243,30 @@ class Sentinel:
if process.timer.value == 0:
# only need to terminate on timeout, otherwise we risk destabilizing
# the queues
task_name = ""
if psutil:
try:
process_name = psutil.Process(process.pid).name()
name_splits = process_name.split(" ")
task_name = name_splits[3] if len(name_splits) >= 4 and name_splits[2] == "processing" else ""
except psutil.NoSuchProcess:
pass
process.terminate()
logger.warning(
_("reincarnated worker %(name)s after timeout")
% {"name": process.name}
)
if task_name:
msg = (
_("reincarnated worker %(name)s after timeout while processing task %(task_name)s")
% {"name": process.name, "task_name": task_name}
)
else:
msg = (
_("reincarnated worker %(name)s after timeout")
% {"name": process.name}
)
logger.critical(msg)
elif int(process.timer.value) == -2:
logger.info(_("recycled worker %(name)s") % {"name": process.name})
else:
logger.error(
logger.critical(
_("reincarnated worker %(name)s after death")
% {"name": process.name}
)
@@ -270,14 +294,14 @@ class Sentinel:
_("%(name)s guarding cluster %(cluster_name)s")
% {
"name": current_process().name,
"cluster_name": humanize(self.cluster_id.hex),
"cluster_name": humanize(self.cluster_id.hex) + f" [{self.queue_name()}]",
}
)
self.start_event.set()
Stat(self).save()
logger.info(
_("Q Cluster %(cluster_name)s running.")
% {"cluster_name": humanize(self.cluster_id.hex)}
% {"cluster_name": humanize(self.cluster_id.hex) + f" [{self.queue_name()}]"}
)
counter = 0
cycle = Conf.GUARD_CYCLE # guard loop sleep in seconds
@@ -359,14 +383,17 @@ def pusher(task_queue: Queue, event: Event, broker: Broker = None):
"""
if not broker:
broker = get_broker()
proc_name = current_process().name
if setproctitle:
setproctitle.setproctitle(f"qcluster {proc_name} pusher")
logger.info(
_("%(process_name)s pushing tasks at %(id)s")
% {"process_name": current_process().name, "id": current_process().pid}
_("%(name)s pushing tasks at %(id)s")
% {"name": proc_name, "id": current_process().pid}
)
while True:
try:
task_set = broker.dequeue()
except Exception as e:
except Exception:
logger.exception("Failed to pull task from broker")
# broker probably crashed. Let the sentinel handle it.
sleep(10)
@@ -377,10 +404,11 @@ def pusher(task_queue: Queue, event: Event, broker: Broker = None):
# unpack the task
try:
task = SignedPackage.loads(task[1])
except (TypeError, BadSignature) as e:
except (TypeError, BadSignature):
logger.exception("Failed to push task to queue")
broker.fail(ack_id)
continue
task["cluster"] = Conf.CLUSTER_NAME # save actual cluster name to orm task table
task["ack_id"] = ack_id
task_queue.put(task)
logger.debug(
@@ -399,9 +427,11 @@ def monitor(result_queue: Queue, broker: Broker = None):
"""
if not broker:
broker = get_broker()
name = current_process().name
proc_name = current_process().name
if setproctitle:
setproctitle.setproctitle(f"qcluster {proc_name} monitor")
logger.info(
_("%(name)s monitoring at %(id)s") % {"name": name, "id": current_process().pid}
_("%(name)s monitoring at %(id)s") % {"name": proc_name, "id": current_process().pid}
)
for task in iter(result_queue.get, "STOP"):
# save the result
@@ -433,7 +463,7 @@ def monitor(result_queue: Queue, broker: Broker = None):
"task_result": task["result"],
}
)
logger.info(_("%(name)s stopped monitoring results") % {"name": name})
logger.info(_("%(name)s stopped monitoring results") % {"name": proc_name})
def worker(
@@ -452,6 +482,9 @@ def worker(
_("%(proc_name)s ready for work at %(id)s")
% {"proc_name": proc_name, "id": current_process().pid}
)
post_spawn.send(sender="django_q", proc_name=proc_name)
if setproctitle:
setproctitle.setproctitle(f"qcluster {proc_name} idle")
task_count = 0
if timeout is None:
timeout = -1
@@ -460,20 +493,33 @@ def worker(
result = None
timer.value = -1 # Idle
task_count += 1
f = task["func"]
# Log task creation and set process name
# Get the function from the task
func = task["func"]
func_name = get_func_repr(func)
logger.info(
_("%(proc_name)s processing '%(func_name)s' (%(task_name)s)")
func_name = get_func_repr(f)
task_name = task["name"]
task_desc = (
_("%(proc_name)s processing %(task_name)s '%(func_name)s'")
% {
"proc_name": proc_name,
"func_name": func_name,
"task_name": task["name"],
"task_name": task_name,
}
)
f = task["func"]
if "group" in task:
task_desc += f" [{task['group']}]"
logger.info(task_desc)
if setproctitle:
proc_title = f"qcluster {proc_name} processing {task_name} '{func_name}'"
if "group" in task:
proc_title += f" [{task['group']}]"
setproctitle.setproctitle(proc_title)
# if it's not an instance try to get it from the string
if not callable(task["func"]):
if not callable(f):
# locate() returns None if f cannot be loaded
f = pydoc.locate(f)
close_old_django_connections()
timer_value = task.pop("timeout", timeout)
@@ -481,22 +527,19 @@ def worker(
pre_execute.send(sender="django_q", func=f, task=task)
# execute the payload
timer.value = timer_value # Busy
try:
if f is None:
# raise a meaningfull error if task["func"] is not a valid function
raise ValueError(f"Function {task['func']} is not defined")
res = f(*task["args"], **task["kwargs"])
result = (res, True)
except Exception:
result = (
_(
"Could not process '%(func_name)s'. Check the location of the "
"function and the args/kwargs."
)
% {"func_name": func_name},
False,
)
except Exception as e:
result = (f"{e} : {traceback.format_exc()}", False)
if error_reporter:
error_reporter.report()
if task.get("sync", False):
raise Exception(result)
raise
with timer.get_lock():
# Process result
task["result"] = result[0]
@@ -504,6 +547,8 @@ def worker(
task["stopped"] = timezone.now()
result_queue.put(task)
timer.value = -1 # Idle
if setproctitle:
setproctitle.setproctitle(f"qcluster {proc_name} idle")
# Recycle
if task_count == Conf.RECYCLE or rss_check():
timer.value = -2 # Recycled
@@ -544,12 +589,7 @@ def save_task(task, broker: Broker):
value = get_func_repr(value)
filters[Conf.SAVE_LIMIT_PER] = value
database_to_use = (
{"using": Conf.ORM if Conf.ORM else Schedule.objects.db}
if not Conf.HAS_REPLICA
else {}
)
with db.transaction.atomic(**database_to_use):
with db.transaction.atomic(using=db.router.db_for_write(Success)):
last = Success.objects.filter(**filters).select_for_update().last()
if (
task["success"]
@@ -584,6 +624,7 @@ def save_task(task, broker: Broker):
hook=task.get("hook"),
args=task["args"],
kwargs=task["kwargs"],
cluster=task.get("cluster"),
started=task["started"],
stopped=task["stopped"],
result=task["result"],
@@ -591,8 +632,8 @@ def save_task(task, broker: Broker):
success=task["success"],
attempt_count=1,
)
except Exception as e:
logger.error(e)
except Exception:
logger.exception("Could not save task result")
def save_cached(task, broker: Broker):
@@ -643,8 +684,8 @@ def save_cached(task, broker: Broker):
)
# save the task
broker.cache.set(task_key, SignedPackage.dumps(task), timeout)
except Exception as e:
logger.error(e)
except Exception:
logger.exception("Could not save task result")
def scheduler(broker: Broker = None):
@@ -655,18 +696,16 @@ def scheduler(broker: Broker = None):
broker = get_broker()
close_old_django_connections()
try:
database_to_use = (
{"using": Conf.ORM if Conf.ORM else Schedule.objects.db}
if not Conf.HAS_REPLICA
else {}
)
with db.transaction.atomic(**database_to_use):
# Only default cluster will handler schedule with default(null) cluster
Q_default = db.models.Q(cluster__isnull=True) if Conf.CLUSTER_NAME == Conf.PREFIX else db.models.Q(pk__in=[])
with db.transaction.atomic(using=db.router.db_for_write(Schedule)):
for s in (
Schedule.objects.select_for_update()
.exclude(repeats=0)
.filter(next_run__lt=timezone.now())
.filter(
db.models.Q(cluster__isnull=True) | db.models.Q(cluster=Conf.PREFIX)
Q_default | db.models.Q(cluster=Conf.CLUSTER_NAME)
)
):
args = ()
@@ -694,52 +733,25 @@ def scheduler(broker: Broker = None):
if type(args) != tuple:
args = (args,)
q_options = kwargs.get("q_options", {})
if s.intended_date_kwarg:
kwargs[s.intended_date_kwarg] = s.next_run.isoformat()
if s.hook:
q_options["hook"] = s.hook
# set up the next run time
if s.schedule_type != s.ONCE:
next_run = s.next_run
while True:
if s.schedule_type == s.MINUTES:
next_run = next_run + timedelta(minutes=(s.minutes or 1))
elif s.schedule_type == s.HOURLY:
next_run = next_run + timedelta(hours=1)
elif s.schedule_type == s.DAILY:
next_run = next_run + timedelta(days=1)
elif s.schedule_type == s.WEEKLY:
next_run = next_run + timedelta(weeks=1)
elif s.schedule_type == s.BIWEEKLY:
next_run = next_run + timedelta(weeks=2)
elif s.schedule_type == s.MONTHLY:
next_run = add_months(next_run, 1)
elif s.schedule_type == s.BIMONTHLY:
next_run = add_months(next_run, 2)
elif s.schedule_type == s.QUARTERLY:
next_run = add_months(next_run, 3)
elif s.schedule_type == s.YEARLY:
next_run = add_years(next_run, 1)
elif s.schedule_type == s.CRON:
if not croniter:
raise ImportError(
_(
"Please install croniter to enable cron "
"expressions"
)
)
next_run = croniter(s.cron, localtime()).get_next(datetime)
next_run = s.calculate_next_run(next_run)
if Conf.CATCH_UP or next_run > localtime():
break
s.next_run = next_run
s.repeats += -1
# send it to the cluster
scheduled_broker = broker
try:
scheduled_broker = get_broker(q_options["broker_name"])
except: # noqa: E722
# invalid broker_name or non existing broker with broker_name
pass
q_options["broker"] = scheduled_broker
# send it to the cluster; any cluster name is allowed in multi-queue scenarios
# because `broker_name` is confusing, using `cluster` name is recommended and takes precedence
q_options["cluster"] = s.cluster or q_options.get("cluster", q_options.pop("broker_name", None))
if q_options['cluster'] is None or q_options['cluster'] == Conf.CLUSTER_NAME:
q_options["broker"] = broker
q_options["group"] = q_options.get("group", s.name or s.id)
kwargs["q_options"] = q_options
s.task = django_q.tasks.async_task(s.func, *args, **kwargs)
@@ -758,11 +770,12 @@ def scheduler(broker: Broker = None):
else:
logger.info(
_(
"%(process_name)s created a task from schedule "
"%(process_name)s created task %(task_name)s from schedule "
"[%(schedule)s]"
)
% {
"process_name": current_process().name,
"task_name": humanize(s.task),
"schedule": s.name or s.id,
}
)
@@ -775,8 +788,8 @@ def scheduler(broker: Broker = None):
s.repeats = 0
# save the schedule
s.save()
except Exception as e:
logger.error(e)
except Exception:
logger.exception("Could not create task from schedule")
def close_old_django_connections():
@@ -842,10 +855,3 @@ def rss_check():
elif psutil:
return psutil.Process().memory_info().rss >= Conf.MAX_RSS * 1024
return False
def localtime() -> datetime:
"""Override for timezone.localtime to deal with naive times and local times"""
if settings.USE_TZ:
return timezone.localtime()
return datetime.now()

View File

@@ -27,6 +27,11 @@ try:
except ModuleNotFoundError:
resource = None
try:
import setproctitle
except ModuleNotFoundError:
setproctitle = None
class Conf:
"""
@@ -34,10 +39,22 @@ class Conf:
"""
try:
conf = settings.Q_CLUSTER
conf = settings.Q_CLUSTER.copy()
except AttributeError:
conf = {}
_Q_CLUSTER_NAME = os.getenv("Q_CLUSTER_NAME")
if _Q_CLUSTER_NAME and _Q_CLUSTER_NAME != conf.get("name") and \
_Q_CLUSTER_NAME != conf.get("cluster_name"):
conf["cluster_name"] = _Q_CLUSTER_NAME
alt_conf = conf.pop("ALT_CLUSTERS")
if isinstance(alt_conf, dict):
alt_conf = alt_conf.get(_Q_CLUSTER_NAME)
if isinstance(alt_conf, dict):
alt_conf.pop('name', None)
alt_conf.pop('cluster_name', None)
conf.update(alt_conf)
# Redis server configuration . Follows standard redis keywords
REDIS = conf.get("redis", {})
@@ -54,9 +71,6 @@ class Conf:
# ORM broker
ORM = conf.get("orm", None)
# ORM support for read/write replicas
HAS_REPLICA = conf.get("has_replica", False)
# Custom broker class
BROKER_CLASS = conf.get("broker_class", None)
@@ -68,8 +82,14 @@ class Conf:
MONGO_DB = conf.get("mongo_db", None)
# Name of the cluster or site. For when you run multiple sites on one redis server
# It's also the `salt` for signing OrmQ, and part of the Redis stats caching key
# For all clusters in one site, PREFIX should be the same value to be able to decrypt payloads
PREFIX = conf.get("name", "default")
# Support alternative cluster name to use multiple queues in one site.
# cluster name and queue name are interchangeable, same thing.
CLUSTER_NAME = conf.get("cluster_name", PREFIX)
# Log output level
LOG_LEVEL = conf.get("log_level", "INFO")
@@ -211,6 +231,11 @@ class Conf:
# to manage workarounds during testing
TESTING = conf.get("testing", False)
# Timezone for next_run, overrules Django timezone
TIME_ZONE = None
if settings.USE_TZ:
TIME_ZONE = conf.get("time_zone", settings.TIME_ZONE)
# logger
logger = logging.getLogger("django-q")

View File

@@ -37,7 +37,7 @@ def loads(
"""
# TimestampSigner.unsign() returns str but base64 and zlib compression
# operate on bytes.
base64d = force_bytes(TimestampSigner(key, salt=salt).unsign(s, max_age=max_age))
base64d = force_bytes(TimestampSigner(key=key, salt=salt).unsign(s, max_age=max_age))
decompress = False
if base64d[:1] == b".":
# It's compressed; uncompress it first

View File

@@ -6,7 +6,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2022-11-12 01:47+0000\n"
"POT-Creation-Date: 2023-01-26 01:38+0000\n"
"PO-Revision-Date: 2018-08-05 18:28+0200\n"
"Last-Translator: Jonas Winkler\n"
"Language-Team: \n"
@@ -17,177 +17,176 @@ msgstr ""
"X-Generator: Poedit 2.1.1\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"
#: django_q/admin.py:43
#: admin.py:43
msgid "Resubmit selected tasks to queue"
msgstr "Ausgewählte Aufgaben erneut ausführen"
#: django_q/admin.py:98 django_q/models.py:228
#: admin.py:107 models.py:293
#, fuzzy
#| msgid "Success"
msgid "success"
msgstr "erfolg"
#: django_q/admin.py:109 django_q/models.py:230
#: admin.py:119 models.py:295
msgid "last_run"
msgstr ""
#: django_q/cluster.py:77
#: cluster.py:79
#, python-format
msgid "Q Cluster %(name)s starting."
msgstr "Q-Cluster %(name)s wird gestartet."
#: django_q/cluster.py:85
#: cluster.py:87
#, fuzzy, python-format
#| msgid "Q Cluster-{} stopping."
msgid "Q Cluster %(name)s stopping."
msgstr "Q-Cluster {name} wird gestoppt."
#: django_q/cluster.py:88
#: cluster.py:90
#, python-format
msgid "Q Cluster %(name)s has stopped."
msgstr "Q-Cluster %(name)s wurde gestoppt."
#: django_q/cluster.py:96
#: cluster.py:97
#, python-format
msgid "%(name)s got signal %(signal)s"
msgstr "%(name)s erhielt das Signal %(signal)s"
#: django_q/cluster.py:219
#: cluster.py:224
#, python-format
msgid "reincarnated monitor %(name)s after sudden death"
msgstr "Monitor %(name)s wurde nach unerwartetem Absturz neu gestartet"
#: django_q/cluster.py:222
#: cluster.py:230
#, python-format
msgid "reincarnated pusher %(name)s after sudden death"
msgstr "Pusher %(name)s wurde nach unerwartetem Absturz neu gestartet"
#: django_q/cluster.py:229
#: cluster.py:250
#, fuzzy, python-format
#| msgid "reincarnated worker %(name)s after timeout"
msgid ""
"reincarnated worker %(name)s after timeout while processing task "
"%(task_name)s"
msgstr "Worker %(name)s wurde nach Zeitüberschreitung neu gestartet"
#: cluster.py:255
#, python-format
msgid "reincarnated worker %(name)s after timeout"
msgstr "Worker %(name)s wurde nach Zeitüberschreitung neu gestartet"
#: django_q/cluster.py:231
#: cluster.py:260
#, python-format
msgid "recycled worker %(name)s"
msgstr "Worker %(name)s wurde wiederverwendet"
#: django_q/cluster.py:233
#: cluster.py:263
#, python-format
msgid "reincarnated worker %(name)s after death"
msgstr "Worker %(name)s wurde nach unerwartetem Absturz neu gestartet"
#: django_q/cluster.py:256
#: cluster.py:287
#, python-format
msgid "%(name)s guarding cluster %(cluster_name)s"
msgstr "%(name)s beschützt das Cluster %(cluster_name)s"
#: django_q/cluster.py:261
#: cluster.py:296
#, python-format
msgid "Q Cluster %(cluster_name)s running."
msgstr "Q-Cluster %(cluster_name)s läuft."
#: django_q/cluster.py:295
#: cluster.py:332
#, python-format
msgid "%(name)s stopping cluster processes"
msgstr "%(name)s hält Cluster-Prozesse an"
#: django_q/cluster.py:320
#: cluster.py:357
#, python-format
msgid "%(name)s waiting for the monitor."
msgstr "%(name)s wartet auf den Monitor."
#: django_q/cluster.py:342
#, python-format
msgid "%(process_name)s pushing tasks at %(id)s"
#: cluster.py:383
#, fuzzy, python-format
#| msgid "%(process_name)s pushing tasks at %(id)s"
msgid "%(name)s pushing tasks at %(id)s"
msgstr "%(process_name)s veröffentlicht Aufagaben auf %(id)s"
#: django_q/cluster.py:363
#: cluster.py:407
#, python-format
msgid "queueing from %(list_key)s"
msgstr "Einreihen von %(list_key)s"
#: django_q/cluster.py:366
#: cluster.py:411
#, python-format
msgid "%(name)s stopped pushing tasks"
msgstr "%(name)s veröffentlicht keine Aufgaben mehr"
#: django_q/cluster.py:378
#: cluster.py:426
#, python-format
msgid "%(name)s monitoring at %(id)s"
msgstr "%(name)s beobachtet auf %(id)s"
#: django_q/cluster.py:395
#: cluster.py:445
#, python-format
msgid "Processed '%(info_name)s' (%(task_name)s)"
msgstr "[%(task_name)s] - '%(info_name)s' wurde verarbeitet"
#: django_q/cluster.py:398
#: cluster.py:451
#, python-format
msgid "Failed '%(info_name)s' (%(task_name)s) - %(task_result)s"
msgstr "'%(info_name)s' (%(task_name)s) ist fehlgeschlagen - %(task_result)s"
#: django_q/cluster.py:399
#: cluster.py:458
#, python-format
msgid "%(name)s stopped monitoring results"
msgstr "%(name)s überwacht keine Ergebnisse mehr"
#: django_q/cluster.py:413
#: cluster.py:474
#, python-format
msgid "%(proc_name)s ready for work at %(id)s"
msgstr "%(proc_name)s ist bereit für Arbeit auf %(id)s"
#: django_q/cluster.py:425
#, python-format
msgid "%(proc_name)s processing '%(func_name)s' (%(task_name)s)"
#: cluster.py:494
#, fuzzy, python-format
#| msgid "%(proc_name)s processing '%(func_name)s' (%(task_name)s)"
msgid "%(proc_name)s processing %(task_name)s '%(func_name)s'"
msgstr "%(proc_name)s verarbeitet '%(func_name)s' (%(task_name)s)"
#: django_q/cluster.py:440
#, python-format
msgid ""
"Could not process '%(func_name)s'. Check the location of the function and "
"the args/kwargs."
msgstr ""
"Konnte '%(func_name)s' nicht verarbeiten. Überprüfen Sie den Ort der "
"Funktion und die args/kwargs."
#: django_q/cluster.py:456
#: cluster.py:546
#, python-format
msgid "%(proc_name)s stopped doing work"
msgstr "%(proc_name)s hat die Arbeit beendet"
#: django_q/cluster.py:649 django_q/models.py:144
msgid "Please install croniter to enable cron expressions"
msgstr "Bitte installieren Sie croniter, um Cron-Ausdrücke zu aktivieren"
#: django_q/cluster.py:672
#: cluster.py:751
#, python-format
msgid "%(process_name)s failed to create a task from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s konnte keine Aufgabe von Zeitplan [%(schedule)s] erstellen"
#: django_q/cluster.py:678
#, python-format
msgid "%(process_name)s created a task from schedule [%(schedule)s]"
#: cluster.py:762
#, fuzzy, python-format
#| msgid "%(process_name)s created a task from schedule [%(schedule)s]"
msgid ""
"%(process_name)s created task %(task_name)s from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s hat eine Aufgabe des Zeitplans [%(schedule)s] erstellt"
#: django_q/cluster.py:718
#: cluster.py:808
msgid "Skipping cpu affinity because psutil was not found."
msgstr "Cpu-Affinität wird übersprungen, da psutil nicht gefunden wurde."
#: django_q/cluster.py:723
#: cluster.py:813
msgid "Faking cpu affinity because it is not supported on this platform"
msgstr ""
"Vortäuschen von CPU-Affinität, da diese auf dieser Plattform nicht "
"unterstützt wird"
#: django_q/cluster.py:744
#: cluster.py:835
#, python-format
msgid "%(pid)s will use cpu %(affinity)s"
msgstr "%(pid)s wird CPU %(affinity)s benutzen"
#: django_q/conf.py:85
#: conf.py:90
#, python-format
msgid ""
"SAVE_LIMIT_PER (%(option)s) is not a valid option. Options are: 'group', "
@@ -197,291 +196,307 @@ msgstr ""
"'group', 'name', 'func' und None. Standard ist None."
#. Translators: Cluster status descriptions
#: django_q/conf.py:194
#: conf.py:207
msgid "Starting"
msgstr "Wird gestartet"
#: django_q/conf.py:195
#: conf.py:208
msgid "Working"
msgstr "Arbeitet"
#: django_q/conf.py:196
#: conf.py:209
msgid "Idle"
msgstr "Leerlauf"
#: django_q/conf.py:197
#: conf.py:210
msgid "Stopped"
msgstr "Gestoppt"
#: django_q/conf.py:198
#: conf.py:211
msgid "Stopping"
msgstr "Wird gestoppt"
#. Translators: help text for qcluster management command
#: django_q/management/commands/qcluster.py:9
#: management/commands/qcluster.py:9
msgid "Starts a Django Q Cluster."
msgstr "Startet ein Django-Q-Cluster."
#. Translators: help text for qinfo management command
#: django_q/management/commands/qinfo.py:11
#: management/commands/qinfo.py:11
msgid "General information over all clusters."
msgstr "Allgemeine Informationen über alle Cluster"
#. Translators: help text for qmemory management command
#: django_q/management/commands/qmemory.py:9
#: management/commands/qmemory.py:9
msgid "Monitors Q Cluster memory usage"
msgstr "Überwacht die Speichernutzung von Q Cluster"
#. Translators: help text for qmonitor management command
#: django_q/management/commands/qmonitor.py:9
#: management/commands/qmonitor.py:9
msgid "Monitors Q Cluster activity"
msgstr "Q-Cluster aktiv überwachen"
#: django_q/models.py:119
#: models.py:125
msgid "Successful task"
msgstr "Erfolgreiche Aufgabe"
#: django_q/models.py:120
#: models.py:126
msgid "Successful tasks"
msgstr "Erfolgreiche Aufgaben"
#: django_q/models.py:135
#: models.py:141
msgid "Failed task"
msgstr "Fehlgeschlagene Aufgabe"
#: django_q/models.py:136
#: models.py:142
msgid "Failed tasks"
msgstr "Fehlgeschlagene Aufgaben"
#: django_q/models.py:160
#: models.py:150 models.py:234
msgid "Please install croniter to enable cron expressions"
msgstr "Bitte installieren Sie croniter, um Cron-Ausdrücke zu aktivieren"
#: models.py:170
msgid "e.g. 1, 2, 'John'"
msgstr "zum Beispiel 1, 2, 'John'"
#: django_q/models.py:162
#: models.py:172
msgid "e.g. x=1, y=2, name='John'"
msgstr "zum Beispiel x=1, y=2, name='John'"
#: django_q/models.py:176
#: models.py:186
msgid "Once"
msgstr "Einmal"
#: django_q/models.py:177
#: models.py:187
msgid "Minutes"
msgstr "Minuten"
#: django_q/models.py:178
#: models.py:188
msgid "Hourly"
msgstr "Stündlich"
#: django_q/models.py:179
#: models.py:189
msgid "Daily"
msgstr "Täglich"
#: django_q/models.py:180
#: models.py:190
msgid "Weekly"
msgstr "Wöchentlich"
#: django_q/models.py:181
#: models.py:191
msgid "Biweekly"
msgstr "Zweiwöchentlich"
#: django_q/models.py:182
#: models.py:192
msgid "Monthly"
msgstr "Monatlich"
#: django_q/models.py:183
#: models.py:193
msgid "Bimonthly"
msgstr "Zweimonatlich"
#: django_q/models.py:184
#: models.py:194
msgid "Quarterly"
msgstr "Vierteljährlich"
#: django_q/models.py:185
#: models.py:195
msgid "Yearly"
msgstr "Jährlich"
#: django_q/models.py:186
#: models.py:196
msgid "Cron"
msgstr "Cron"
#: django_q/models.py:189
#: models.py:199
msgid "Schedule Type"
msgstr "Zeitplan-Typ"
#: django_q/models.py:192
#: models.py:202
msgid "Number of minutes for the Minutes type"
msgstr "Anzahl Minuten für den Typ 'Minuten'"
#: django_q/models.py:195
#: models.py:205
msgid "Repeats"
msgstr "Wiederhohlungen"
#: django_q/models.py:195
#: models.py:205
msgid "n = n times, -1 = forever"
msgstr "n = n mal, -1 = für immer"
#: django_q/models.py:198
#: models.py:208
msgid "Next Run"
msgstr "Nächste Ausführung"
#: django_q/models.py:205
#: models.py:215
msgid "Cron expression"
msgstr "Cron-Ausdruck"
#: django_q/models.py:235
#: models.py:224
msgid "Name of kwarg to pass intended schedule date"
msgstr "Name des zu passierenden Kwargs vorgesehenes Datum"
#: models.py:299
msgid "Scheduled task"
msgstr "Geplante Aufgabe"
#: django_q/models.py:236
#: models.py:300
msgid "Scheduled tasks"
msgstr "Geplante Aufgaben"
#: django_q/models.py:262
#: models.py:326
msgid "Queued task"
msgstr "Eingereihte Aufgabe"
#: django_q/models.py:263
#: models.py:327
msgid "Queued tasks"
msgstr "Eingereihte Aufgaben"
#: django_q/monitor.py:62 django_q/monitor.py:339
#: monitor.py:64 monitor.py:348
msgid "Host"
msgstr "Host"
#: django_q/monitor.py:66 django_q/monitor.py:343 django_q/monitor.py:450
#: monitor.py:68 monitor.py:352 monitor.py:459
msgid "Id"
msgstr "Id"
#: django_q/monitor.py:70
#: monitor.py:72
msgid "State"
msgstr "Status"
#: django_q/monitor.py:74
#: monitor.py:76
msgid "Pool"
msgstr "Pool"
#: django_q/monitor.py:78
#: monitor.py:80
msgid "TQ"
msgstr "TQ"
#: django_q/monitor.py:82
#: monitor.py:84
msgid "RQ"
msgstr "RQ"
#: django_q/monitor.py:86
#: monitor.py:88
msgid "RC"
msgstr "RC"
#: django_q/monitor.py:90
#: monitor.py:92
msgid "Up"
msgstr "Up"
#: django_q/monitor.py:170 django_q/monitor.py:279
#: monitor.py:172 monitor.py:286
msgid "Queued"
msgstr "Eingereiht"
#: django_q/monitor.py:178
#: monitor.py:180
msgid "Success"
msgstr "Erfolg"
#: django_q/monitor.py:188 django_q/monitor.py:287
#: monitor.py:190 monitor.py:294
msgid "Failures"
msgstr "Fehlschläge"
#: django_q/monitor.py:199 django_q/monitor.py:485
#: monitor.py:201 monitor.py:498
msgid "[Press q to quit]"
msgstr "[Drücken Sie q zum Beenden]"
#: django_q/monitor.py:223
#: monitor.py:227
msgid "day"
msgstr "Tag"
#: django_q/monitor.py:244
#: monitor.py:248
msgid "second"
msgstr "Sekunde"
#: django_q/monitor.py:247
#: monitor.py:251
msgid "minute"
msgstr "Minute"
#: django_q/monitor.py:250
#: monitor.py:254
msgid "hour"
msgstr "Stunde"
#: django_q/monitor.py:260
#: monitor.py:263
#, python-format
msgid "-- %(prefix)s %(version)s on %(info)s --"
msgstr "-- %(prefix)s %(version)s auf %(info)s --"
#: django_q/monitor.py:266
#: monitor.py:273
msgid "Clusters"
msgstr "Cluster"
#: django_q/monitor.py:270
#: monitor.py:277
msgid "Workers"
msgstr "Arbeiter"
#: django_q/monitor.py:274
#: monitor.py:281
msgid "Restarts"
msgstr "Neustarts"
#: django_q/monitor.py:283
#: monitor.py:290
msgid "Successes"
msgstr "Erfolge"
#: django_q/monitor.py:292
#: monitor.py:299
msgid "Schedules"
msgstr "Zeitpläne"
#: django_q/monitor.py:296
#: monitor.py:303
#, python-format
msgid "Tasks/%(per)s"
msgstr "Aufgaben/%(per)s"
#: django_q/monitor.py:300
#: monitor.py:307
msgid "Avg time"
msgstr "Durchschnittl. Zeit"
#: django_q/monitor.py:348
#: monitor.py:357
msgid "Available (%)"
msgstr "Verfügbar (%)"
#: django_q/monitor.py:354
#: monitor.py:363
msgid "Available (MB)"
msgstr "Verfügbar (MB)"
#: django_q/monitor.py:359
#: monitor.py:368
msgid "Total (MB)"
msgstr "Insgesamt (MB)"
#: django_q/monitor.py:364
#: monitor.py:373
msgid "Sentinel (MB)"
msgstr "Sentinel (MB)"
#: django_q/monitor.py:370
#: monitor.py:379
msgid "Monitor (MB)"
msgstr "Monitor (MB)"
#: django_q/monitor.py:376
#: monitor.py:385
msgid "Workers (MB)"
msgstr "Arbeiter (MB)"
#: django_q/monitor.py:478
#: monitor.py:487
#, python-format
msgid "Available lowest (): %(memory_percent)s ((at)s)"
msgstr "Niedrigste verfügbar (): %(memory_percent)s ((at)s)"
#: django_q/monitor.py:496
#: monitor.py:509
msgid "No clusters appear to be running."
msgstr "Es scheinen keine Cluster zu laufen."
#: django_q/signals.py:22
#: signals.py:22
#, python-format
msgid "malformed return hook '%(hook)s' for [%(name)s]"
msgstr "Ungültiger Return-Hook '%(hook)s' für [%(name)s]"
#: django_q/signals.py:30
#: signals.py:30
#, python-format
msgid "return hook %(hook)s failed on [%(name)s] because %(error)s"
msgstr "Return-Hook %(hook)s für [%(name)s] ist gescheitert: %(error)s"
#, python-format
#~ msgid ""
#~ "Could not process '%(func_name)s'. Check the location of the function and "
#~ "the args/kwargs."
#~ msgstr ""
#~ "Konnte '%(func_name)s' nicht verarbeiten. Überprüfen Sie den Ort der "
#~ "Funktion und die args/kwargs."

View File

@@ -6,7 +6,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2022-11-12 01:47+0000\n"
"POT-Creation-Date: 2023-01-26 01:38+0000\n"
"PO-Revision-Date: 2018-08-05 18:28+0200\n"
"Last-Translator: Thierry BOULOGNE <contact@tng-concepts.com>\n"
"Language-Team: \n"
@@ -17,176 +17,175 @@ msgstr ""
"X-Generator: Poedit 2.1.1\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"
#: django_q/admin.py:43
#: admin.py:43
msgid "Resubmit selected tasks to queue"
msgstr "Resoumettre les tâches sélectionnées à la file d'attente"
#: django_q/admin.py:98 django_q/models.py:228
#: admin.py:107 models.py:293
#, fuzzy
#| msgid "Success"
msgid "success"
msgstr "succès"
#: django_q/admin.py:109 django_q/models.py:230
#: admin.py:119 models.py:295
msgid "last_run"
msgstr ""
#: django_q/cluster.py:77
#: cluster.py:79
#, python-format
msgid "Q Cluster %(name)s starting."
msgstr "Démarrage de Q Cluster-%(name)s."
#: django_q/cluster.py:85
#: cluster.py:87
#, python-format
msgid "Q Cluster %(name)s stopping."
msgstr "Arrêt de Q Cluster-%(name)s."
#: django_q/cluster.py:88
#: cluster.py:90
#, python-format
msgid "Q Cluster %(name)s has stopped."
msgstr "Q Cluster-%(name)s a été arrêté."
#: django_q/cluster.py:96
#: cluster.py:97
#, python-format
msgid "%(name)s got signal %(signal)s"
msgstr "%(name)s à reçu le signal %(signal)s"
#: django_q/cluster.py:219
#: cluster.py:224
#, python-format
msgid "reincarnated monitor %(name)s after sudden death"
msgstr "moniteur réintégré %(name)s après un arrêt intempestif"
msgstr "surveillant %(name)s réincarné après un arrêt intempestif"
#: django_q/cluster.py:222
#: cluster.py:230
#, python-format
msgid "reincarnated pusher %(name)s after sudden death"
msgstr "pousseur réintégré %(name)s après un arrêt intempestif"
msgstr "répartiteur %(name)s réincarné après un arrêt intempestif"
#: django_q/cluster.py:229
#: cluster.py:250
#, python-format
msgid ""
"reincarnated worker %(name)s after timeout while processing task "
"%(task_name)s"
msgstr ""
"processus %(name)s réincarné, délai de traitement dépassé pour la tâche "
"%(task_name)s"
#: cluster.py:255
#, python-format
msgid "reincarnated worker %(name)s after timeout"
msgstr "processus réintégré %(name)s après un arrêt une attente trop longue"
msgstr "processus %(name)s réincarné, délai de traitement dépassé"
#: django_q/cluster.py:231
#: cluster.py:260
#, python-format
msgid "recycled worker %(name)s"
msgstr "processus recyclé %(name)s"
#: django_q/cluster.py:233
#: cluster.py:263
#, python-format
msgid "reincarnated worker %(name)s after death"
msgstr "processus réintégré %(name)s après arrêt"
#: django_q/cluster.py:256
#: cluster.py:287
#, python-format
msgid "%(name)s guarding cluster %(cluster_name)s"
msgstr "%(name)s surveillance du cluster à %(cluster_name)s"
#: django_q/cluster.py:261
#: cluster.py:296
#, python-format
msgid "Q Cluster %(cluster_name)s running."
msgstr "Démarrage de Q Cluster-%(cluster_name)s."
#: django_q/cluster.py:295
#: cluster.py:332
#, python-format
msgid "%(name)s stopping cluster processes"
msgstr "%(name)s arrêt des processus de cluster"
msgstr "%(name)s arrêt des processus du cluster"
#: django_q/cluster.py:320
#: cluster.py:357
#, python-format
msgid "%(name)s waiting for the monitor."
msgstr "%(name)s en attente du moniteur."
msgstr "%(name)s en attente du surveillant."
#: django_q/cluster.py:342
#: cluster.py:383
#, python-format
msgid "%(process_name)s pushing tasks at %(id)s"
msgstr "%(process_name)s tâche envoyé à %(id)s"
msgid "%(name)s pushing tasks at %(id)s"
msgstr "%(name)s répartit les tâches %(id)s"
#: django_q/cluster.py:363
#: cluster.py:407
#, python-format
msgid "queueing from %(list_key)s"
msgstr "mise en file d'attente de %(list_key)s"
#: django_q/cluster.py:366
#: cluster.py:411
#, python-format
msgid "%(name)s stopped pushing tasks"
msgstr "%(name)s a cessé de pousser les tâches"
msgstr "%(name)s a cessé de répartir les tâches"
#: django_q/cluster.py:378
#: cluster.py:426
#, python-format
msgid "%(name)s monitoring at %(id)s"
msgstr "%(name)s Surveillance de %(id)s"
msgstr "%(name)s surveille les résultats %(id)s"
#: django_q/cluster.py:395
#: cluster.py:445
#, python-format
msgid "Processed '%(info_name)s' (%(task_name)s)"
msgstr "traité '%(info_name)s' (%(task_name)s)"
#: django_q/cluster.py:398
#: cluster.py:451
#, python-format
msgid "Failed '%(info_name)s' (%(task_name)s) - %(task_result)s"
msgstr "Manqué '%(info_name)s' (%(task_name)s) - %(task_result)s"
#: django_q/cluster.py:399
#: cluster.py:458
#, python-format
msgid "%(name)s stopped monitoring results"
msgstr "%(name)s arrêt des résultats de surveillance"
msgstr "%(name)s a cessé de de surveiller les résultats"
#: django_q/cluster.py:413
#: cluster.py:474
#, python-format
msgid "%(proc_name)s ready for work at %(id)s"
msgstr "%(proc_name)s prêt pour le travail à %(id)s"
#: django_q/cluster.py:425
#, python-format
msgid "%(proc_name)s processing '%(func_name)s' (%(task_name)s)"
msgstr "%(proc_name)s en traitement '%(func_name)s' (%(task_name)s)"
#: cluster.py:494
#, fuzzy, python-format
msgid "%(proc_name)s processing %(task_name)s '%(func_name)s'"
msgstr "%(proc_name)s exécute %(task_name)s '%(func_name)s'"
#: django_q/cluster.py:440
#, python-format
msgid ""
"Could not process '%(func_name)s'. Check the location of the function and "
"the args/kwargs."
msgstr ""
"Impossible de traiter '%(func_name)s'. Vérifiez l'emplacement de la fonction "
"et les args/kwargs."
#: django_q/cluster.py:456
#: cluster.py:546
#, python-format
msgid "%(proc_name)s stopped doing work"
msgstr "%(proc_name)s arrêté de travailler"
msgstr "%(proc_name)s a cessé de travailler"
#: django_q/cluster.py:649 django_q/models.py:144
msgid "Please install croniter to enable cron expressions"
msgstr "Veuillez installer croniter pour activer les expressions croniques."
#: django_q/cluster.py:672
#: cluster.py:751
#, python-format
msgid "%(process_name)s failed to create a task from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s Echec de la création d'une tâche à partir de Schedule "
"[%(schedule)s]"
#: django_q/cluster.py:678
#: cluster.py:762
#, python-format
msgid "%(process_name)s created a task from schedule [%(schedule)s]"
msgstr "%(process_name)s a créé une tâche à partir de Schedule [%(schedule)s]"
msgid ""
"%(process_name)s created task %(task_name)s from schedule [%(schedule)s]"
msgstr ""
"%(process_name)s a créé la tâche %(task_name)s à partir de Schedule "
"[%(schedule)s]"
#: django_q/cluster.py:718
#: cluster.py:808
msgid "Skipping cpu affinity because psutil was not found."
msgstr "Sauter l'affinité du processeur parce que psutil n'a pas été trouvé."
msgstr "L'affinité cpu ne sera pas définie car psutil n'a pas été trouvé."
#: django_q/cluster.py:723
#: cluster.py:813
msgid "Faking cpu affinity because it is not supported on this platform"
msgstr ""
"Simulation de l'affinité du processeur parce qu'elle n'est pas supportée sur "
"cette plateforme."
"Simulation de l'affinité cpu parce qu'elle n'est pas supportée sur cette "
"plateforme."
#: django_q/cluster.py:744
#: cluster.py:835
#, python-format
msgid "%(pid)s will use cpu %(affinity)s"
msgstr "%(pid)s utilisera le CPU %(affinity)s"
#: django_q/conf.py:85
#: conf.py:90
#, python-format
msgid ""
"SAVE_LIMIT_PER (%(option)s) is not a valid option. Options are: 'group', "
@@ -196,299 +195,315 @@ msgstr ""
"'group', 'name', 'func' et None. La valeur par défaut est None."
#. Translators: Cluster status descriptions
#: django_q/conf.py:194
#: conf.py:207
msgid "Starting"
msgstr "Démarrage"
#: django_q/conf.py:195
#: conf.py:208
msgid "Working"
msgstr "Actif"
#: django_q/conf.py:196
#: conf.py:209
msgid "Idle"
msgstr "En attente"
#: django_q/conf.py:197
#: conf.py:210
msgid "Stopped"
msgstr "Arrêté"
#: django_q/conf.py:198
#: conf.py:211
msgid "Stopping"
msgstr "En cours darrêt"
#. Translators: help text for qcluster management command
#: django_q/management/commands/qcluster.py:9
#: management/commands/qcluster.py:9
msgid "Starts a Django Q Cluster."
msgstr "Démarre un cluster Django Q."
#. Translators: help text for qinfo management command
#: django_q/management/commands/qinfo.py:11
#: management/commands/qinfo.py:11
msgid "General information over all clusters."
msgstr "Informations générales sur tous les clusters."
#. Translators: help text for qmemory management command
#: django_q/management/commands/qmemory.py:9
#: management/commands/qmemory.py:9
#, fuzzy
#| msgid "Monitors Q Cluster activity"
msgid "Monitors Q Cluster memory usage"
msgstr "Surveille l'utilisation de la mémoire du cluster Q"
msgstr "Surveille l'utilisation mémoire du Q cluster"
#. Translators: help text for qmonitor management command
#: django_q/management/commands/qmonitor.py:9
#: management/commands/qmonitor.py:9
msgid "Monitors Q Cluster activity"
msgstr "Activité du cluster Moniteur Q"
msgstr "Surveille l'activité de Q cluster"
#: django_q/models.py:119
#: models.py:125
msgid "Successful task"
msgstr "Tâche réussie"
#: django_q/models.py:120
#: models.py:126
msgid "Successful tasks"
msgstr "Tâches réussies"
#: django_q/models.py:135
#: models.py:141
msgid "Failed task"
msgstr "Tâche échoué"
#: django_q/models.py:136
#: models.py:142
msgid "Failed tasks"
msgstr "Tâches échouées"
#: django_q/models.py:160
#: models.py:150 models.py:234
msgid "Please install croniter to enable cron expressions"
msgstr "Veuillez installer croniter pour activer les expressions cron."
#: models.py:170
msgid "e.g. 1, 2, 'John'"
msgstr "ex. 1, 2, Jean"
#: django_q/models.py:162
#: models.py:172
msgid "e.g. x=1, y=2, name='John'"
msgstr "p. ex. x = 1, y = 2, Nom = Jean"
#: django_q/models.py:176
#: models.py:186
msgid "Once"
msgstr "Une fois"
#: django_q/models.py:177
#: models.py:187
msgid "Minutes"
msgstr "Minutes"
#: django_q/models.py:178
#: models.py:188
msgid "Hourly"
msgstr "Toutes les heures"
#: django_q/models.py:179
#: models.py:189
msgid "Daily"
msgstr "Quotidien"
#: django_q/models.py:180
#: models.py:190
msgid "Weekly"
msgstr "Hebdomadaire"
#: django_q/models.py:181
#: models.py:191
#, fuzzy
#| msgid "Weekly"
msgid "Biweekly"
msgstr "Bihebdomadaire"
#: django_q/models.py:182
#: models.py:192
msgid "Monthly"
msgstr "Mensuel"
#: django_q/models.py:183
#: models.py:193
#, fuzzy
#| msgid "Monthly"
msgid "Bimonthly"
msgstr "Bimestriel"
#: django_q/models.py:184
#: models.py:194
msgid "Quarterly"
msgstr "Tous les quart-dheure"
#: django_q/models.py:185
#: models.py:195
msgid "Yearly"
msgstr "Annuel"
#: django_q/models.py:186
#: models.py:196
msgid "Cron"
msgstr "Cron"
#: django_q/models.py:189
#: models.py:199
msgid "Schedule Type"
msgstr "Type de plannification"
#: django_q/models.py:192
#: models.py:202
msgid "Number of minutes for the Minutes type"
msgstr "Nombre de minutes pour le type de minutes"
#: django_q/models.py:195
#: models.py:205
msgid "Repeats"
msgstr "Répéter"
#: django_q/models.py:195
#: models.py:205
msgid "n = n times, -1 = forever"
msgstr "n = n fois,-1 = Toujours"
#: django_q/models.py:198
#: models.py:208
msgid "Next Run"
msgstr "Prochaine exécution"
#: django_q/models.py:205
#: models.py:215
msgid "Cron expression"
msgstr "Expression du Cron"
msgstr "Expression Cron"
#: django_q/models.py:235
#: models.py:224
msgid "Name of kwarg to pass intended schedule date"
msgstr "Nom du kwarg à passer Date prévue de l'horaire"
#: models.py:299
msgid "Scheduled task"
msgstr "Tâche planifiée"
#: django_q/models.py:236
#: models.py:300
msgid "Scheduled tasks"
msgstr "Tâches planifiées"
#: django_q/models.py:262
#: models.py:326
msgid "Queued task"
msgstr "Tâche en file d'attente"
#: django_q/models.py:263
#: models.py:327
msgid "Queued tasks"
msgstr "Tâches en file d'attente"
#: django_q/monitor.py:62 django_q/monitor.py:339
#: monitor.py:64 monitor.py:348
msgid "Host"
msgstr "Hôte"
#: django_q/monitor.py:66 django_q/monitor.py:343 django_q/monitor.py:450
#: monitor.py:68 monitor.py:352 monitor.py:459
msgid "Id"
msgstr "Id"
#: django_q/monitor.py:70
#: monitor.py:72
msgid "State"
msgstr "Statut"
#: django_q/monitor.py:74
#: monitor.py:76
msgid "Pool"
msgstr "Piscine"
#: django_q/monitor.py:78
#: monitor.py:80
msgid "TQ"
msgstr "TQ"
#: django_q/monitor.py:82
#: monitor.py:84
msgid "RQ"
msgstr "RQ"
#: django_q/monitor.py:86
#: monitor.py:88
msgid "RC"
msgstr "RC"
#: django_q/monitor.py:90
#: monitor.py:92
msgid "Up"
msgstr "Haut"
#: django_q/monitor.py:170 django_q/monitor.py:279
#: monitor.py:172 monitor.py:286
msgid "Queued"
msgstr "En file d'attente"
#: django_q/monitor.py:178
#: monitor.py:180
msgid "Success"
msgstr "Succès"
#: django_q/monitor.py:188 django_q/monitor.py:287
#: monitor.py:190 monitor.py:294
msgid "Failures"
msgstr "Défaillances"
#: django_q/monitor.py:199 django_q/monitor.py:485
#: monitor.py:201 monitor.py:498
msgid "[Press q to quit]"
msgstr "[appuyez sur q pour quitter]"
#: django_q/monitor.py:223
#: monitor.py:227
msgid "day"
msgstr "jour"
#: django_q/monitor.py:244
#: monitor.py:248
msgid "second"
msgstr "seconde"
#: django_q/monitor.py:247
#: monitor.py:251
msgid "minute"
msgstr "minute"
#: django_q/monitor.py:250
#: monitor.py:254
msgid "hour"
msgstr "heure"
#: django_q/monitor.py:260
#: monitor.py:263
#, python-format
msgid "-- %(prefix)s %(version)s on %(info)s --"
msgstr "--%(prefix)s %(version)s sur %(info)s --"
#: django_q/monitor.py:266
#: monitor.py:273
msgid "Clusters"
msgstr "Grappes"
#: django_q/monitor.py:270
#: monitor.py:277
msgid "Workers"
msgstr "Processus"
#: django_q/monitor.py:274
#: monitor.py:281
msgid "Restarts"
msgstr "Redémarrages"
#: django_q/monitor.py:283
#: monitor.py:290
msgid "Successes"
msgstr "Succès"
#: django_q/monitor.py:292
#: monitor.py:299
msgid "Schedules"
msgstr "Planifications"
#: django_q/monitor.py:296
#: monitor.py:303
#, python-format
msgid "Tasks/%(per)s"
msgstr "Tâches/%(per)s"
#: django_q/monitor.py:300
#: monitor.py:307
msgid "Avg time"
msgstr "Temps Moyen"
#: django_q/monitor.py:348
#: monitor.py:357
msgid "Available (%)"
msgstr ""
#: django_q/monitor.py:354
#: monitor.py:363
msgid "Available (MB)"
msgstr "Disponible sur (MB)"
#: django_q/monitor.py:359
#: monitor.py:368
msgid "Total (MB)"
msgstr "Total (MB)"
#: django_q/monitor.py:364
#: monitor.py:373
msgid "Sentinel (MB)"
msgstr "Sentinel (MB)"
#: django_q/monitor.py:370
#: monitor.py:379
msgid "Monitor (MB)"
msgstr "Monitor (MB)"
#: django_q/monitor.py:376
#: monitor.py:385
#, fuzzy
#| msgid "Workers"
msgid "Workers (MB)"
msgstr "Processus (MB)"
#: django_q/monitor.py:478
#: monitor.py:487
#, python-format
msgid "Available lowest (): %(memory_percent)s ((at)s)"
msgstr "Disponible le plus bas () : %(memory_percent)s ((at)s)"
#: django_q/monitor.py:496
#: monitor.py:509
msgid "No clusters appear to be running."
msgstr "Aucun cluster ne semble être en cours d'exécution."
#: django_q/signals.py:22
#: signals.py:22
#, python-format
msgid "malformed return hook '%(hook)s' for [%(name)s]"
msgstr "hook de retour mal formé' %(hook)s 'pour [%(name)s]"
msgstr "hook de retour '%(hook)s' mal formé pour [%(name)s]"
#: django_q/signals.py:30
#: signals.py:30
#, python-format
msgid "return hook %(hook)s failed on [%(name)s] because %(error)s"
msgstr "le crochet de retour %(hook)s a échoué sur [%(name)s] parce que %(error)s"
msgstr "hook de retour %(hook)s a échoué sur [%(name)s] à cause de %(error)s"
#, python-format
#~ msgid ""
#~ "Could not process '%(func_name)s'. Check the location of the function and "
#~ "the args/kwargs."
#~ msgstr ""
#~ "Impossible de traiter '%(func_name)s'. Vérifiez l'emplacement de la "
#~ "fonction et les args/kwargs."

View File

@@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2022-11-12 01:47+0000\n"
"POT-Creation-Date: 2023-01-26 01:38+0000\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: Ethem Güner <ethemguener@gmail.com>\n"
"Language-Team: \n"
@@ -18,172 +18,171 @@ msgstr ""
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"
#: django_q/admin.py:43
#: admin.py:43
msgid "Resubmit selected tasks to queue"
msgstr "Seçili işleri kuyruğa tekrar gönder"
#: django_q/admin.py:98 django_q/models.py:228
#: admin.py:107 models.py:293
#, fuzzy
#| msgid "Success"
msgid "success"
msgstr "başarılı olanlar"
#: django_q/admin.py:109 django_q/models.py:230
#: admin.py:119 models.py:295
msgid "last_run"
msgstr ""
#: django_q/cluster.py:77
#: cluster.py:79
#, python-format
msgid "Q Cluster %(name)s starting."
msgstr "Q Cluster %(name)s başlatılıyor."
#: django_q/cluster.py:85
#: cluster.py:87
#, python-format
msgid "Q Cluster %(name)s stopping."
msgstr "Q Cluster %(name)s durduruluyor."
#: django_q/cluster.py:88
#: cluster.py:90
#, python-format
msgid "Q Cluster %(name)s has stopped."
msgstr "Q Cluster %(name)s durduruldu."
#: django_q/cluster.py:96
#: cluster.py:97
#, python-format
msgid "%(name)s got signal %(signal)s"
msgstr "%(name)s, %(signal)s pid'inde izleniyor/monitoring yapılıyor."
#: django_q/cluster.py:219
#: cluster.py:224
#, python-format
msgid "reincarnated monitor %(name)s after sudden death"
msgstr "Monitor %(name)s ani ölüm sonrası tekrar dirildi"
#: django_q/cluster.py:222
#: cluster.py:230
#, python-format
msgid "reincarnated pusher %(name)s after sudden death"
msgstr "Pusher %(name)s ani ölüm sonrası tekrar dirildi"
#: django_q/cluster.py:229
#: cluster.py:250
#, fuzzy, python-format
#| msgid "reincarnated worker %(name)s after timeout"
msgid ""
"reincarnated worker %(name)s after timeout while processing task "
"%(task_name)s"
msgstr "Worker %(name)s zaman aşımı sonrası tekrar dirildi"
#: cluster.py:255
#, python-format
msgid "reincarnated worker %(name)s after timeout"
msgstr "Worker %(name)s zaman aşımı sonrası tekrar dirildi"
#: django_q/cluster.py:231
#: cluster.py:260
#, python-format
msgid "recycled worker %(name)s"
msgstr "Worker %(name)s geri döndürüldü"
#: django_q/cluster.py:233
#: cluster.py:263
#, python-format
msgid "reincarnated worker %(name)s after death"
msgstr "Worker %(name)s ani ölüm sonrası tekrar dirildi"
#: django_q/cluster.py:256
#: cluster.py:287
#, python-format
msgid "%(name)s guarding cluster %(cluster_name)s"
msgstr "%(name)s, %(cluster_name)s cluster'ını koruyor"
#: django_q/cluster.py:261
#: cluster.py:296
#, python-format
msgid "Q Cluster %(cluster_name)s running."
msgstr "Q Cluster %(cluster_name)s başlatılıyor."
#: django_q/cluster.py:295
#: cluster.py:332
#, python-format
msgid "%(name)s stopping cluster processes"
msgstr "Cluster %(name)s işlemleri durduruluyor."
#: django_q/cluster.py:320
#: cluster.py:357
#, python-format
msgid "%(name)s waiting for the monitor."
msgstr "%(name)s monitor için bekliyor."
#: django_q/cluster.py:342
#, python-format
msgid "%(process_name)s pushing tasks at %(id)s"
#: cluster.py:383
#, fuzzy, python-format
#| msgid "%(process_name)s pushing tasks at %(id)s"
msgid "%(name)s pushing tasks at %(id)s"
msgstr "%(process_name)s, işleri %(id)s pid'ine gönderiyor."
#: django_q/cluster.py:363
#: cluster.py:407
#, python-format
msgid "queueing from %(list_key)s"
msgstr ""
#: django_q/cluster.py:366
#: cluster.py:411
#, python-format
msgid "%(name)s stopped pushing tasks"
msgstr "%(name)s işleri göndermeyi durdurdu"
#: django_q/cluster.py:378
#: cluster.py:426
#, python-format
msgid "%(name)s monitoring at %(id)s"
msgstr "%(name)s, %(id)s pid'inde izleniyor/monitoring yapılıyor."
#: django_q/cluster.py:395
#: cluster.py:445
#, python-format
msgid "Processed '%(info_name)s' (%(task_name)s)"
msgstr "[%(task_name)s] - '%(info_name)s işlendi."
#: django_q/cluster.py:398
#: cluster.py:451
#, python-format
msgid "Failed '%(info_name)s' (%(task_name)s) - %(task_result)s"
msgstr "[%(task_name)s] - '%(info_name)s' - %(task_result)s başarısız oldu"
#: django_q/cluster.py:399
#: cluster.py:458
#, python-format
msgid "%(name)s stopped monitoring results"
msgstr "%(name)s sonuçları göstermeyi bıraktı"
#: django_q/cluster.py:413
#: cluster.py:474
#, python-format
msgid "%(proc_name)s ready for work at %(id)s"
msgstr "%(proc_name)s, %(id)s pid'inde çalışmaya hazır"
#: django_q/cluster.py:425
#, python-format
msgid "%(proc_name)s processing '%(func_name)s' (%(task_name)s)"
#: cluster.py:494
#, fuzzy, python-format
#| msgid "%(proc_name)s processing '%(func_name)s' (%(task_name)s)"
msgid "%(proc_name)s processing %(task_name)s '%(func_name)s'"
msgstr "%(proc_name)s, '%(func_name)s' [%(task_name)s] işlerini işiyor"
#: django_q/cluster.py:440
#, python-format
msgid ""
"Could not process '%(func_name)s'. Check the location of the function and "
"the args/kwargs."
msgstr ""
"%(func_name)s' işlenemedi. İşlevin konumunu ve args/kwargs öğelerini kontrol "
"edin."
#: django_q/cluster.py:456
#: cluster.py:546
#, python-format
msgid "%(proc_name)s stopped doing work"
msgstr "%(proc_name)s çalışmayı bıraktı"
#: django_q/cluster.py:649 django_q/models.py:144
msgid "Please install croniter to enable cron expressions"
msgstr "Cron expressions'ları açmak için croniter yükleyin"
#: django_q/cluster.py:672
#: cluster.py:751
#, python-format
msgid "%(process_name)s failed to create a task from schedule [%(schedule)s]"
msgstr "%(process_name)s programdan bir görev oluşturamadı [%(schedule)s]"
#: django_q/cluster.py:678
#, python-format
msgid "%(process_name)s created a task from schedule [%(schedule)s]"
#: cluster.py:762
#, fuzzy, python-format
#| msgid "%(process_name)s created a task from schedule [%(schedule)s]"
msgid ""
"%(process_name)s created task %(task_name)s from schedule [%(schedule)s]"
msgstr "%(process_name)s programdan bir görev oluşturamadı [%(schedule)s]"
#: django_q/cluster.py:718
#: cluster.py:808
msgid "Skipping cpu affinity because psutil was not found."
msgstr "Psutil bulunamadığı için cpu benzeşimi atlanıyor."
#: django_q/cluster.py:723
#: cluster.py:813
msgid "Faking cpu affinity because it is not supported on this platform"
msgstr "Bu platformda desteklenmediği için sahte cpu benzeşimi"
#: django_q/cluster.py:744
#: cluster.py:835
#, python-format
msgid "%(pid)s will use cpu %(affinity)s"
msgstr "%(pid)s cpu %(affinity)s kullanacaktır"
#: django_q/conf.py:85
#: conf.py:90
#, python-format
msgid ""
"SAVE_LIMIT_PER (%(option)s) is not a valid option. Options are: 'group', "
@@ -193,295 +192,311 @@ msgstr ""
"'group', 'name', 'func' ve None. Varsayılan değer None'dır."
#. Translators: Cluster status descriptions
#: django_q/conf.py:194
#: conf.py:207
msgid "Starting"
msgstr "Başlıyor"
#: django_q/conf.py:195
#: conf.py:208
msgid "Working"
msgstr "Çalışıyor"
#: django_q/conf.py:196
#: conf.py:209
msgid "Idle"
msgstr "Boşta"
#: django_q/conf.py:197
#: conf.py:210
msgid "Stopped"
msgstr "Durdu"
#: django_q/conf.py:198
#: conf.py:211
msgid "Stopping"
msgstr "Durduruluyor"
#. Translators: help text for qcluster management command
#: django_q/management/commands/qcluster.py:9
#: management/commands/qcluster.py:9
msgid "Starts a Django Q Cluster."
msgstr "Bir Django Q Cluster çalıştırır."
#. Translators: help text for qinfo management command
#: django_q/management/commands/qinfo.py:11
#: management/commands/qinfo.py:11
msgid "General information over all clusters."
msgstr "Tüm cluster'lar için genel bilgiler."
#. Translators: help text for qmemory management command
#: django_q/management/commands/qmemory.py:9
#: management/commands/qmemory.py:9
msgid "Monitors Q Cluster memory usage"
msgstr "Q Cluster'ın bellek kullanımını izler"
#. Translators: help text for qmonitor management command
#: django_q/management/commands/qmonitor.py:9
#: management/commands/qmonitor.py:9
msgid "Monitors Q Cluster activity"
msgstr "Q Cluster'ın aktivitelerini izler"
#: django_q/models.py:119
#: models.py:125
msgid "Successful task"
msgstr "Başarılı iş"
#: django_q/models.py:120
#: models.py:126
msgid "Successful tasks"
msgstr "Başarılı işler"
#: django_q/models.py:135
#: models.py:141
msgid "Failed task"
msgstr "Başarısız iş"
#: django_q/models.py:136
#: models.py:142
msgid "Failed tasks"
msgstr "Başarısız işler"
#: django_q/models.py:160
#: models.py:150 models.py:234
msgid "Please install croniter to enable cron expressions"
msgstr "Cron expressions'ları açmak için croniter yükleyin"
#: models.py:170
msgid "e.g. 1, 2, 'John'"
msgstr "Örneğin: 1, 2, 'Melih'"
#: django_q/models.py:162
#: models.py:172
msgid "e.g. x=1, y=2, name='John'"
msgstr "Örneğin: x=1, y=2, name='Melih'"
#: django_q/models.py:176
#: models.py:186
msgid "Once"
msgstr "Bir kere"
#: django_q/models.py:177
#: models.py:187
msgid "Minutes"
msgstr "Dakika"
#: django_q/models.py:178
#: models.py:188
msgid "Hourly"
msgstr "Saatlik"
#: django_q/models.py:179
#: models.py:189
msgid "Daily"
msgstr "Günlük"
#: django_q/models.py:180
#: models.py:190
msgid "Weekly"
msgstr "Haftalık"
#: django_q/models.py:181
#: models.py:191
#, fuzzy
#| msgid "Weekly"
msgid "Biweekly"
msgstr "İki haftada bir"
#: django_q/models.py:182
#: models.py:192
msgid "Monthly"
msgstr "Aylık"
#: django_q/models.py:183
#: models.py:193
#, fuzzy
#| msgid "Monthly"
msgid "Bimonthly"
msgstr "İki ayda bir"
#: django_q/models.py:184
#: models.py:194
msgid "Quarterly"
msgstr "Bir Çeyrek (3 Ay)"
#: django_q/models.py:185
#: models.py:195
msgid "Yearly"
msgstr "Yıllık"
#: django_q/models.py:186
#: models.py:196
msgid "Cron"
msgstr ""
#: django_q/models.py:189
#: models.py:199
msgid "Schedule Type"
msgstr "Zamanlama Tipi"
#: django_q/models.py:192
#: models.py:202
msgid "Number of minutes for the Minutes type"
msgstr "Dakika tipine göre dakika sayısı"
#: django_q/models.py:195
#: models.py:205
msgid "Repeats"
msgstr "Tekrar eder"
#: django_q/models.py:195
#: models.py:205
msgid "n = n times, -1 = forever"
msgstr "n = n kere, -1 = sonsuza kadar"
#: django_q/models.py:198
#: models.py:208
msgid "Next Run"
msgstr "Bir dahaki çalışma tarihi"
#: django_q/models.py:205
#: models.py:215
msgid "Cron expression"
msgstr ""
#: django_q/models.py:235
#: models.py:224
msgid "Name of kwarg to pass intended schedule date"
msgstr "Geçilecek kwarg'ın adı öngörülen program tarihi"
#: models.py:299
msgid "Scheduled task"
msgstr "Zamanlanmış iş"
#: django_q/models.py:236
#: models.py:300
msgid "Scheduled tasks"
msgstr "Zamanlanmış işler"
#: django_q/models.py:262
#: models.py:326
msgid "Queued task"
msgstr "Sıraya alınmış iş"
#: django_q/models.py:263
#: models.py:327
msgid "Queued tasks"
msgstr "Sıraya alınmış işler"
#: django_q/monitor.py:62 django_q/monitor.py:339
#: monitor.py:64 monitor.py:348
msgid "Host"
msgstr ""
#: django_q/monitor.py:66 django_q/monitor.py:343 django_q/monitor.py:450
#: monitor.py:68 monitor.py:352 monitor.py:459
msgid "Id"
msgstr ""
#: django_q/monitor.py:70
#: monitor.py:72
msgid "State"
msgstr "Durum"
#: django_q/monitor.py:74
#: monitor.py:76
msgid "Pool"
msgstr "Havuz"
#: django_q/monitor.py:78
#: monitor.py:80
msgid "TQ"
msgstr ""
#: django_q/monitor.py:82
#: monitor.py:84
msgid "RQ"
msgstr ""
#: django_q/monitor.py:86
#: monitor.py:88
msgid "RC"
msgstr ""
#: django_q/monitor.py:90
#: monitor.py:92
msgid "Up"
msgstr ""
#: django_q/monitor.py:170 django_q/monitor.py:279
#: monitor.py:172 monitor.py:286
msgid "Queued"
msgstr "Sıraya alınmış"
#: django_q/monitor.py:178
#: monitor.py:180
msgid "Success"
msgstr "Başarılı olanlar"
#: django_q/monitor.py:188 django_q/monitor.py:287
#: monitor.py:190 monitor.py:294
msgid "Failures"
msgstr "Başarısız olanlar"
#: django_q/monitor.py:199 django_q/monitor.py:485
#: monitor.py:201 monitor.py:498
msgid "[Press q to quit]"
msgstr "[Çıkmak için q'ya basın]"
#: django_q/monitor.py:223
#: monitor.py:227
msgid "day"
msgstr "gün"
#: django_q/monitor.py:244
#: monitor.py:248
msgid "second"
msgstr "saniye"
#: django_q/monitor.py:247
#: monitor.py:251
msgid "minute"
msgstr "dakika"
#: django_q/monitor.py:250
#: monitor.py:254
msgid "hour"
msgstr "saat"
#: django_q/monitor.py:260
#: monitor.py:263
#, python-format
msgid "-- %(prefix)s %(version)s on %(info)s --"
msgstr "-- %(prefix)s %(version)s üzerinde %(info)s --"
#: django_q/monitor.py:266
#: monitor.py:273
msgid "Clusters"
msgstr ""
#: django_q/monitor.py:270
#: monitor.py:277
msgid "Workers"
msgstr ""
#: django_q/monitor.py:274
#: monitor.py:281
msgid "Restarts"
msgstr "Yeniden çalıştırmalar"
#: django_q/monitor.py:283
#: monitor.py:290
msgid "Successes"
msgstr "Başarılı olanlar"
#: django_q/monitor.py:292
#: monitor.py:299
msgid "Schedules"
msgstr "Zamanlanmışlar"
#: django_q/monitor.py:296
#: monitor.py:303
#, python-format
msgid "Tasks/%(per)s"
msgstr "İş/%(per)s"
#: django_q/monitor.py:300
#: monitor.py:307
msgid "Avg time"
msgstr "Ortalama süre"
#: django_q/monitor.py:348
#: monitor.py:357
msgid "Available (%)"
msgstr "Müsait (%) "
#: django_q/monitor.py:354
#: monitor.py:363
msgid "Available (MB)"
msgstr "Müsait (MB)"
#: django_q/monitor.py:359
#: monitor.py:368
msgid "Total (MB)"
msgstr "Toplam (MB)"
#: django_q/monitor.py:364
#: monitor.py:373
msgid "Sentinel (MB)"
msgstr ""
#: django_q/monitor.py:370
#: monitor.py:379
msgid "Monitor (MB)"
msgstr "İzleme (MB)"
#: django_q/monitor.py:376
#: monitor.py:385
msgid "Workers (MB)"
msgstr ""
#: django_q/monitor.py:478
#: monitor.py:487
#, python-format
msgid "Available lowest (): %(memory_percent)s ((at)s)"
msgstr "Mevcut en düşük (): %(memory_percent)s ((at)s)"
#: django_q/monitor.py:496
#: monitor.py:509
msgid "No clusters appear to be running."
msgstr "Hiçbir küme çalışıyor görünmüyor."
#: django_q/signals.py:22
#: signals.py:22
#, python-format
msgid "malformed return hook '%(hook)s' for [%(name)s]"
msgstr ""
#: django_q/signals.py:30
#: signals.py:30
#, python-format
msgid "return hook %(hook)s failed on [%(name)s] because %(error)s"
msgstr ""
#, python-format
#~ msgid ""
#~ "Could not process '%(func_name)s'. Check the location of the function and "
#~ "the args/kwargs."
#~ msgstr ""
#~ "%(func_name)s' işlenemedi. İşlevin konumunu ve args/kwargs öğelerini "
#~ "kontrol edin."

View File

@@ -2,6 +2,7 @@ from django.core.management.base import BaseCommand
from django.utils.translation import gettext as _
from django_q.cluster import Cluster
import os
class Command(BaseCommand):
@@ -16,8 +17,21 @@ class Command(BaseCommand):
default=False,
help="Run once and then stop.",
)
parser.add_argument(
"-n",
"--name",
dest="cluster_name",
default=None,
help="Set alternative cluster name instead of the name in Q_CLUSTER settings (for multi-queue setup). "
"On Linux you should set name through `Q_CLUSTER_NAME=cluster_name python manage.py qcluster` instead."
)
def handle(self, *args, **options):
# Set alternative cluster_name before creating the cluster (cluster_name is broker's queue_name, too)
cluster_name = options.get("cluster_name")
if cluster_name:
os.environ["Q_CLUSTER_NAME"] = cluster_name
q = Cluster()
q.start()
if options.get("run_once", False):

View File

@@ -0,0 +1,25 @@
# Generated by Django 4.1.2 on 2023-01-15 22:34
from django.db import migrations, models
import django_q.models
class Migration(migrations.Migration):
dependencies = [
("django_q", "0015_alter_schedule_schedule_type"),
]
operations = [
migrations.AddField(
model_name="schedule",
name="intended_date_kwarg",
field=models.CharField(
blank=True,
help_text="Name of kwarg to pass intended schedule date",
max_length=100,
null=True,
validators=[django_q.models.validate_kwarg],
),
),
]

View File

@@ -0,0 +1,43 @@
# Generated by Django 4.1.5 on 2023-03-07 12:18
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("django_q", "0016_schedule_intended_date_kwarg"),
]
operations = [
migrations.AddField(
model_name="task",
name="cluster",
field=models.CharField(blank=True, default=None, max_length=100, null=True),
),
migrations.AlterField(
model_name="ormq",
name="key",
field=models.CharField(
help_text="Name of the target cluster", max_length=100
),
),
migrations.AlterField(
model_name="ormq",
name="lock",
field=models.DateTimeField(
help_text="Prevent any cluster from pulling until", null=True
),
),
migrations.AlterField(
model_name="schedule",
name="cluster",
field=models.CharField(
blank=True,
default=None,
help_text="Name of the target cluster",
max_length=100,
null=True,
),
),
]

View File

@@ -1,3 +1,6 @@
from datetime import datetime, timedelta
from keyword import iskeyword
# Django
from django import get_version
from django.core.exceptions import ValidationError
@@ -5,16 +8,19 @@ from django.db import models
from django.template.defaultfilters import truncatechars
from django.urls import reverse
from django.utils import timezone
from django.utils.timezone import is_aware
from django.utils.html import format_html
from django.utils.translation import gettext_lazy as _
from django.utils.functional import cached_property
# External
from picklefield import PickledObjectField
from picklefield.fields import dbsafe_decode
# Local
from django_q.conf import croniter
from django_q.conf import croniter, Conf
from django_q.signing import SignedPackage
from django_q.utils import localtime, add_months, add_years
from .utils import get_func_repr
@@ -28,6 +34,7 @@ class Task(models.Model):
kwargs = PickledObjectField(null=True, protocol=-1)
result = PickledObjectField(null=True, protocol=-1)
group = models.CharField(max_length=100, editable=False, null=True)
cluster = models.CharField(max_length=100, default=None, null=True, blank=True)
started = models.DateTimeField(editable=False)
stopped = models.DateTimeField(editable=False)
success = models.BooleanField(default=True, editable=False)
@@ -149,6 +156,10 @@ def validate_cron(value):
raise ValidationError(e)
def validate_kwarg(value):
return value.isidentifier() and not iskeyword(value)
class Schedule(models.Model):
name = models.CharField(max_length=100, null=True, blank=True)
func = models.CharField(max_length=256, help_text="e.g. module.tasks.function")
@@ -206,7 +217,65 @@ class Schedule(models.Model):
help_text=_("Cron expression"),
)
task = models.CharField(max_length=100, null=True, editable=False)
cluster = models.CharField(max_length=100, default=None, null=True, blank=True)
cluster = models.CharField(
max_length=100, default=None, null=True, blank=True,
help_text=_("Name of the target cluster")
)
intended_date_kwarg = models.CharField(
max_length=100,
null=True,
blank=True,
validators=[validate_kwarg],
help_text=_("Name of kwarg to pass intended schedule date"),
)
def calculate_next_run(self, next_run=None):
# next run is always in UTC
next_run = next_run or self.next_run
if self.schedule_type == self.CRON:
if not croniter:
raise ImportError(
_("Please install croniter to enable cron expressions")
)
return croniter(self.cron, localtime()).get_next(datetime)
if self.schedule_type == self.MINUTES:
add = timedelta(minutes=(self.minutes or 1))
elif self.schedule_type == self.HOURLY:
add = timedelta(hours=1)
elif self.schedule_type == self.DAILY:
add = timedelta(days=1)
elif self.schedule_type == self.WEEKLY:
add = timedelta(weeks=1)
elif self.schedule_type == self.BIWEEKLY:
add = timedelta(weeks=2)
elif self.schedule_type == self.MONTHLY:
add = timedelta(days=(add_months(next_run, 1) - next_run).days)
elif self.schedule_type == self.BIMONTHLY:
add = timedelta(days=(add_months(next_run, 2) - next_run).days)
elif self.schedule_type == self.QUARTERLY:
add = timedelta(days=(add_months(next_run, 3) - next_run).days)
elif self.schedule_type == self.YEARLY:
add = timedelta(days=(add_years(next_run, 1) - next_run).days)
# add normal timedelta, we will correct this later based on timezone
next_run += add
# DST differencers don't matter with minutes, hourly or yearly, so skip those
if self.schedule_type not in [self.MINUTES, self.HOURLY, self.YEARLY]:
# Get localtimes and then remove the tzinfo, so we can get the actual difference
current_next_run = localtime(next_run - add).replace(tzinfo=None)
new_next_run = localtime(next_run).replace(tzinfo=None)
# get the difference between them, this should be (-)1 or (-)0.5 hour
# based on DST active or not
extra_diff = (new_next_run - current_next_run) - add
# subtract difference
next_run -= extra_diff
return next_run
def success(self):
if self.task and Task.objects.filter(id=self.task):
@@ -238,24 +307,38 @@ class Schedule(models.Model):
class OrmQ(models.Model):
key = models.CharField(max_length=100)
key = models.CharField(max_length=100, help_text=_("Name of the target cluster"))
payload = models.TextField()
lock = models.DateTimeField(null=True)
lock = models.DateTimeField(null=True, help_text=_("Prevent any cluster from pulling until"))
@cached_property
def task(self):
return SignedPackage.loads(self.payload)
try:
return SignedPackage.loads(self.payload)
except Exception as e:
return {"id": "*" + e.__class__.__name__}
def func(self):
return get_func_repr(self.task()["func"])
return get_func_repr(self.task.get("func"))
def task_id(self):
return self.task()["id"]
return self.task.get("id")
def name(self):
return self.task()["name"]
return self.task.get("name")
def group(self):
return self.task().get("group")
return self.task.get("group")
def args(self):
return self.task.get("args")
def kwargs(self):
return self.task.get("kwargs")
def q_options(self):
exclude = {"id", "name", "group", "func", "args", "kwargs"}
return {k: v for k, v in self.task.items() if k not in exclude}
class Meta:
app_label = "django_q"

View File

@@ -31,6 +31,8 @@ def call_hook(sender, instance, **kwargs):
% {"hook": instance.hook, "name": instance.name, "error": str(e)}
)
# args: proc_name
post_spawn = Signal()
# args: task
pre_enqueue = Signal()

View File

@@ -31,6 +31,7 @@ def async_task(func, *args, **kwargs):
"iter_cached",
"chain",
"broker",
"cluster",
"timeout",
)
q_options = keywords.pop("q_options", {})
@@ -52,7 +53,7 @@ def async_task(func, *args, **kwargs):
elif key in keywords:
task[key] = keywords.pop(key)
# don't serialize the broker
broker = task.pop("broker", get_broker())
broker = task.pop("broker", None) or get_broker(task.get("cluster"))
# overrides
if "cached" not in task and Conf.CACHED:
task["cached"] = Conf.CACHED
@@ -71,7 +72,7 @@ def async_task(func, *args, **kwargs):
return _sync(pack)
# push it
enqueue_id = broker.enqueue(pack)
logger.info(f"Enqueued {enqueue_id}")
logger.info(f"Enqueued [{broker.list_key}] {enqueue_id}")
logger.debug(f"Pushed {tag}")
return task["id"]
@@ -90,6 +91,7 @@ def schedule(func, *args, **kwargs):
:type next_run: datetime.datetime
:param cluster: optional cluster name.
:param cron: optional cron expression
:param intended_date_kwarg: optional identifier to pass intended schedule date.
:param kwargs: function keyword arguments.
:return: the schedule object.
:rtype: Schedule
@@ -102,6 +104,7 @@ def schedule(func, *args, **kwargs):
next_run = kwargs.pop("next_run", timezone.now())
cron = kwargs.pop("cron", None)
cluster = kwargs.pop("cluster", None)
intended_date_kwarg = kwargs.pop("intended_date_kwarg", None)
# check for name duplicates instead of am unique constraint
if name and Schedule.objects.filter(name=name).exists():
@@ -120,6 +123,7 @@ def schedule(func, *args, **kwargs):
next_run=next_run,
cron=cron,
cluster=cluster,
intended_date_kwarg=intended_date_kwarg,
)
# make sure we trigger validation
s.full_clean()
@@ -270,6 +274,7 @@ def fetch_cached(task_id, wait=0, broker=None):
hook=task.get("hook"),
args=task["args"],
kwargs=task["kwargs"],
cluster=task.get("cluster"),
started=task["started"],
stopped=task["stopped"],
result=task["result"],
@@ -340,6 +345,7 @@ def fetch_group_cached(group_id, failures=True, wait=0, count=None, broker=None)
hook=task.get("hook"),
args=task["args"],
kwargs=task["kwargs"],
cluster=task.get("cluster"),
started=task["started"],
stopped=task["stopped"],
result=task["result"],

View File

@@ -75,7 +75,7 @@ DATABASES = {
LANGUAGE_CODE = "en-us"
TIME_ZONE = "UTC"
TIME_ZONE = "Europe/Amsterdam"
USE_I18N = True

View File

@@ -64,7 +64,7 @@ def test_admin_views(admin_client, monkeypatch):
# resubmit the failure
url = reverse("admin:django_q_failure_changelist")
data = {"action": "retry_failed", "_selected_action": [f.pk]}
data = {"action": "resubmit_task", "_selected_action": [f.pk]}
response = admin_client.post(url, data)
assert response.status_code == 302
assert Failure.objects.filter(name=f.id).exists() is False
@@ -84,3 +84,10 @@ def test_admin_views(admin_client, monkeypatch):
data = {"post": "yes"}
response = admin_client.post(url, data)
assert response.status_code == 302
# Resubmit a successful task.
url = reverse("admin:django_q_success_changelist")
data = {"action": "resubmit_task", "_selected_action": [t.pk]}
initial_queue_count = OrmQ.objects.count()
response = admin_client.post(url, data)
assert response.status_code == 302
assert OrmQ.objects.count() > initial_queue_count

View File

@@ -29,7 +29,7 @@ from django_q.tasks import (
result,
result_group,
)
from django_q.tests.tasks import multiply
from django_q.tests.tasks import multiply, TaskError
from django_q.utils import add_months, add_years
myPath = os.path.dirname(os.path.abspath(__file__))
@@ -64,7 +64,7 @@ def test_sync(broker):
@pytest.mark.django_db
def test_sync_raise_exception(broker):
with pytest.raises(Exception):
with pytest.raises(TaskError):
async_task("django_q.tests.tasks.raise_exception", broker=broker, sync=True)

View File

@@ -1,9 +1,10 @@
import os
from datetime import timedelta
from datetime import datetime, timedelta
from multiprocessing import Event, Value
from unittest import mock
import pytest
import django
from django.core.exceptions import ValidationError
from django.db import IntegrityError
from django.test import override_settings
@@ -23,6 +24,15 @@ from django_q.tests.testing_utilities.multiple_database_routers import (
)
from django_q.utils import add_months
if django.VERSION < (4, 0):
# pytz is the default in django 3.2. Remove when no support for 3.2
from pytz import timezone as ZoneInfo
else:
try:
from zoneinfo import ZoneInfo
except ImportError:
from backports.zoneinfo import ZoneInfo
@pytest.fixture
def broker(monkeypatch) -> Broker:
@@ -40,25 +50,11 @@ def orm_broker(monkeypatch) -> None:
monkeypatch.setattr(Conf, "ORM", "default")
@pytest.fixture
def orm_no_replica_broker(orm_broker, monkeypatch) -> Broker:
"""Generates a Broker with a disabled read replica database configuration."""
monkeypatch.setattr(Conf, "HAS_REPLICA", False)
return get_broker(list_key="scheduler_test:q")
@pytest.fixture
def orm_replica_broker(orm_broker, monkeypatch) -> Broker:
"""Generates a Broker with read replica database configuration."""
monkeypatch.setattr(Conf, "HAS_REPLICA", True)
return get_broker(list_key="scheduler_test:q")
REPLICA_DATABASE_ROUTERS = [
f"{TestingReplicaDatabaseRouter.__module__}.{TestingReplicaDatabaseRouter.__name__}"
]
REPLICA_DATABASES = {
"default": {
"writable": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": os.path.join(BASE_DIR, "db.sqlite3"),
},
@@ -83,6 +79,109 @@ MULTIPLE_APPS_DATABASES = {
}
@pytest.mark.django_db
def test_scheduler_daylight_saving_time_daily(broker, monkeypatch):
# Set up a startdate in the Amsterdam timezone (without dst 1 hour ahead). The
# 28th of March 2021 is the day when sunlight saving starts (at 2 am)
monkeypatch.setattr(Conf, "TIME_ZONE", "Europe/Amsterdam")
tz = ZoneInfo('Europe/Amsterdam')
broker.list_key = "scheduler_test:q"
# Let's start a schedule at 1 am on the 27th of March. This is in AMS timezone.
# So, 2021-03-27 00:00:00 when saved (due to TZ being Amsterdam and saved in UTC)
start_date = datetime(2021, 3, 27, 1, 0, 0)
# Create schedule with the next run date on the start date. It will move one day
# forward when we run the scheduler
schedule = create_schedule(
"math.copysign",
1,
-1,
name="test math",
schedule_type=Schedule.DAILY,
next_run=start_date,
)
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
# It's now the day after exactly at midnight UTC
next_run = schedule.next_run
assert str(next_run) == "2021-03-28 00:00:00+00:00"
# In the Amsterdam timezone, it's 1 hour over midnight (+01)
next_run = next_run.astimezone(tz)
assert str(next_run) == "2021-03-28 01:00:00+01:00"
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
next_run = schedule.next_run
assert str(next_run) == "2021-03-28 23:00:00+00:00"
next_run = next_run.astimezone(tz)
# In the Amsterdam timezone, it's 1 hour over midnight (+02)
assert str(next_run) == "2021-03-29 01:00:00+02:00"
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
next_run = schedule.next_run
assert str(next_run) == "2021-03-29 23:00:00+00:00"
next_run = next_run.astimezone(tz)
assert str(next_run) == "2021-03-30 01:00:00+02:00"
# Create second schedule with the next run date on the start date. It will move
# one day forward when we run the scheduler
start_date = datetime(2021, 10, 29, 1, 0, 0)
schedule = create_schedule(
"django_q.tests.tasks.word_multiply",
2,
name="multiply",
schedule_type=Schedule.DAILY,
next_run=start_date,
)
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
next_run = schedule.next_run
assert str(next_run) == "2021-10-29 23:00:00+00:00"
# In the Amsterdam timezone, it's 1 hour over midnight (+02)
next_run = next_run.astimezone(tz)
assert str(next_run) == "2021-10-30 01:00:00+02:00"
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
next_run = schedule.next_run
assert str(next_run) == "2021-10-30 23:00:00+00:00"
# In the Amsterdam timezone, it's 1 hour over midnight (+02)
next_run = next_run.astimezone(tz)
assert str(next_run) == "2021-10-31 01:00:00+02:00"
# Run scheduler so we get the next run date
scheduler(broker=broker)
schedule.refresh_from_db()
next_run = schedule.next_run
assert str(next_run) == "2021-11-01 00:00:00+00:00"
# In the Amsterdam timezone, it's 1 hour over midnight (+01)
# Switch of DST
next_run = next_run.astimezone(tz)
assert str(next_run) == "2021-11-01 01:00:00+01:00"
@pytest.mark.django_db
def test_scheduler(broker, monkeypatch):
broker.list_key = "scheduler_test:q"
@@ -255,7 +354,7 @@ def test_scheduler(broker, monkeypatch):
assert schedule.next_run.date() == (timezone.now() + timedelta(weeks=2)).date()
broker.delete_queue()
monkeypatch.setattr(Conf, "PREFIX", "some_cluster_name")
monkeypatch.setattr(Conf, "CLUSTER_NAME", "some_cluster_name")
# create a schedule on another cluster
schedule = create_schedule(
"math.copysign",
@@ -279,7 +378,7 @@ def test_scheduler(broker, monkeypatch):
# queue must be empty
assert task_queue.qsize() == 0
monkeypatch.setattr(Conf, "PREFIX", "default")
monkeypatch.setattr(Conf, "CLUSTER_NAME", "default")
# create a schedule on the same cluster
schedule = create_schedule(
"math.copysign",
@@ -304,43 +403,54 @@ def test_scheduler(broker, monkeypatch):
assert task_queue.qsize() == 1
@override_settings(
DATABASE_ROUTERS=REPLICA_DATABASE_ROUTERS, DATABASES=REPLICA_DATABASES
)
@pytest.mark.django_db
def test_scheduler_atomic_transaction_must_specify_a_database_when_no_replicas_are_used(
orm_no_replica_broker: Broker,
):
"""
GIVEN a environment without a read replica database
WHEN the scheduler is called
THEN the transaction atomic must be called using the configured database in the
Conf.ORM settings.
"""
broker = orm_no_replica_broker
with mock.patch("django_q.cluster.db") as mocked_db:
scheduler(broker=broker)
# The router should correctly set the database to use!
mocked_db.transaction.atomic.assert_called_with(using=broker.connection.db)
def test_intended_schedule_kwarg(broker, monkeypatch):
broker.list_key = "scheduler_test:q"
broker.delete_queue()
run_date = timezone.now()-timedelta(hours=1)
schedule = create_schedule(
"math.copysign",
1,
-1,
name="test math",
hook="django_q.tests.tasks.result",
schedule_type=Schedule.HOURLY,
repeats=1,
next_run=run_date,
intended_date_kwarg='intended_date',
)
assert schedule.last_run() is None
assert schedule.intended_date_kwarg == 'intended_date'
# run scheduler
scheduler(broker=broker)
# set up the workflow
task_queue = Queue()
stop_event = Event()
stop_event.set()
# push it
pusher(task_queue, stop_event, broker=broker)
assert task_queue.qsize() == 1
task = task_queue.get()
assert 'intended_date' in task['kwargs']
assert task['kwargs']['intended_date'] == run_date.isoformat()
@override_settings(
DATABASE_ROUTERS=REPLICA_DATABASE_ROUTERS, DATABASES=REPLICA_DATABASES
)
@pytest.mark.django_db
def test_scheduler_atomic_must_specify_no_db_when_read_write_replicas_are_used(
orm_replica_broker: Broker,
def test_scheduler_atomic_must_specify_the_write_db(
orm_broker: Broker,
):
"""
GIVEN a environment with a read/write configured replica database
WHEN the scheduler is called
THEN the transaction must be called without a specific database, thus letting the
database router pick.
THEN the transaction must be called with the write database.
"""
with mock.patch("django_q.cluster.db") as mocked_db:
scheduler(broker=orm_replica_broker)
# No specific databases should be set here, this is the job of the router!
mocked_db.transaction.atomic.assert_called_with()
broker = get_broker(list_key="scheduler_test:q")
with mock.patch("django_q.cluster.db.transaction") as mocked_db:
scheduler(broker=broker)
mocked_db.atomic.assert_called_with(using="writable")
@override_settings(
@@ -348,20 +458,17 @@ def test_scheduler_atomic_must_specify_no_db_when_read_write_replicas_are_used(
)
@pytest.mark.django_db
def test_scheduler_atomic_must_specify_the_database_based_on_router_redirection(
orm_no_replica_broker: Broker,
orm_broker: Broker,
):
"""
GIVEN a environment without a read replica database
WHEN the scheduler is called
THEN the transaction atomic must be called using the configured database in the
Conf.ORM settings.
THEN the transaction atomic must be called using the default connection.
"""
broker = orm_no_replica_broker
with mock.patch("django_q.cluster.db") as mocked_db:
broker = get_broker(list_key="scheduler_test:q")
with mock.patch("django_q.cluster.db.transaction") as mocked_db:
scheduler(broker=broker)
# The router should correctly set the database to use!
assert broker.connection.db == "default"
mocked_db.transaction.atomic.assert_called_with(using=broker.connection.db)
mocked_db.atomic.assert_called_with(using="default")
def test_localtime():

View File

@@ -12,9 +12,9 @@ class TestingReplicaDatabaseRouter:
def db_for_write(self, model, **hints):
"""
Always write to DEFAULT database
Always write to WRITABLE database
"""
return "default"
return "writable"
class TestingMultipleAppsDatabaseRouter:

View File

@@ -1,7 +1,23 @@
from datetime import datetime
import calendar
import inspect
from datetime import date
import django
from django.utils import timezone
from django.conf import settings
from django_q.conf import Conf
if django.VERSION < (4, 0):
# pytz is the default in django 3.2. Remove when no support for 3.2
from pytz import timezone as ZoneInfo
else:
try:
from zoneinfo import ZoneInfo
except ImportError:
from backports.zoneinfo import ZoneInfo
# credits: https://stackoverflow.com/a/4131114
# Made them aware of timezone
@@ -38,4 +54,21 @@ def get_func_repr(func):
f"{func.__self__.__module__}." f"{func.__self__.__name__}.{func.__name__}"
)
else:
return str(func)
return str(func) if func else None
def localtime(value=None) -> datetime:
"""Override for timezone.localtime to deal with naive times and local times"""
if settings.USE_TZ:
if django.VERSION >= (4, 0) and settings.USE_DEPRECATED_PYTZ:
import pytz
convert_to_tz = pytz.timezone(Conf.TIME_ZONE)
else:
convert_to_tz = ZoneInfo(Conf.TIME_ZONE)
return timezone.localtime(value=value, timezone=convert_to_tz)
if value is None:
return datetime.now()
else:
return value

View File

@@ -11,36 +11,36 @@ Start your cluster using Django's ``manage.py`` command::
You should see the cluster starting ::
10:57:40 [Q] INFO Q Cluster-31781 starting.
10:57:40 [Q] INFO Process-1:1 ready for work at 31784
10:57:40 [Q] INFO Process-1:2 ready for work at 31785
10:57:40 [Q] INFO Process-1:3 ready for work at 31786
10:57:40 [Q] INFO Process-1:4 ready for work at 31787
10:57:40 [Q] INFO Process-1:5 ready for work at 31788
10:57:40 [Q] INFO Process-1:6 ready for work at 31789
10:57:40 [Q] INFO Process-1:7 ready for work at 31790
10:57:40 [Q] INFO Process-1:8 ready for work at 31791
10:57:40 [Q] INFO Process-1:9 monitoring at 31792
10:57:40 [Q] INFO Process-1 guarding cluster at 31783
10:57:40 [Q] INFO Process-1:10 pushing tasks at 31793
10:57:40 [Q] INFO Q Cluster-31781 running.
10:57:40 [Q] INFO Q Cluster freddie-uncle-twenty-ten starting.
10:57:40 [Q] INFO Process-ede257774c4444c980ab479f10947acc ready for work at 31784
10:57:40 [Q] INFO Process-ed580482da3f42968230baa2e4253e42 ready for work at 31785
10:57:40 [Q] INFO Process-8a370dc2bc1d49aa9864e517c9895f74 ready for work at 31786
10:57:40 [Q] INFO Process-74912f9844264d1397c6e54476b530c0 ready for work at 31787
10:57:40 [Q] INFO Process-b00edb26c6074a6189e5696c60aeb35b ready for work at 31788
10:57:40 [Q] INFO Process-b0862965db04479f9784a26639ee51e0 ready for work at 31789
10:57:40 [Q] INFO Process-7e8abbb8ca2d4d9bb20a937dd5e2872b ready for work at 31790
10:57:40 [Q] INFO Process-b0862965db04479f9784a26639ee51e0 ready for work at 31791
10:57:40 [Q] INFO Process-67fa9461ac034736a766cd813f617e62 monitoring at 31792
10:57:40 [Q] INFO Process-eac052c646b2459797cee98bdb84c85d guarding cluster at 31783
10:57:40 [Q] INFO Process-5d98deb19b1e4b2da2ef1e5bd6824f75 pushing tasks at 31793
10:57:40 [Q] INFO Q Cluster freddie-uncle-twenty-ten running.
Stopping the cluster with ctrl-c or either the ``SIGTERM`` and ``SIGKILL`` signals, will initiate the :ref:`stop_procedure`::
16:44:12 [Q] INFO Q Cluster-31781 stopping.
16:44:12 [Q] INFO Process-1 stopping cluster processes
16:44:13 [Q] INFO Process-1:10 stopped pushing tasks
16:44:13 [Q] INFO Process-1:6 stopped doing work
16:44:13 [Q] INFO Process-1:4 stopped doing work
16:44:13 [Q] INFO Process-1:1 stopped doing work
16:44:13 [Q] INFO Process-1:5 stopped doing work
16:44:13 [Q] INFO Process-1:7 stopped doing work
16:44:13 [Q] INFO Process-1:3 stopped doing work
16:44:13 [Q] INFO Process-1:8 stopped doing work
16:44:13 [Q] INFO Process-1:2 stopped doing work
16:44:14 [Q] INFO Process-1:9 stopped monitoring results
16:44:15 [Q] INFO Q Cluster-31781 has stopped.
16:44:12 [Q] INFO Q Cluster freddie-uncle-twenty-ten stopping.
16:44:12 [Q] INFO Process-eac052c646b2459797cee98bdb84c85d stopping cluster processes
16:44:13 [Q] INFO Process-5d98deb19b1e4b2da2ef1e5bd6824f75 stopped pushing tasks
16:44:13 [Q] INFO Process-b0862965db04479f9784a26639ee51e0 stopped doing work
16:44:13 [Q] INFO Process-7e8abbb8ca2d4d9bb20a937dd5e2872b stopped doing work
16:44:13 [Q] INFO Process-b0862965db04479f9784a26639ee51e0 stopped doing work
16:44:13 [Q] INFO Process-b00edb26c6074a6189e5696c60aeb35b stopped doing work
16:44:13 [Q] INFO Process-74912f9844264d1397c6e54476b530c0 stopped doing work
16:44:13 [Q] INFO Process-8a370dc2bc1d49aa9864e517c9895f74 stopped doing work
16:44:13 [Q] INFO Process-ed580482da3f42968230baa2e4253e42 stopped doing work
16:44:13 [Q] INFO Process-ede257774c4444c980ab479f10947acc stopped doing work
16:44:14 [Q] INFO Process-67fa9461ac034736a766cd813f617e62 stopped monitoring results
16:44:15 [Q] INFO Q Cluster freddie-uncle-twenty-ten has stopped.
The number of workers, optional timeouts, recycles and cpu_affinity can be controlled via the :doc:`configure` settings.
@@ -52,6 +52,30 @@ You can have multiple clusters on multiple machines, working on the same queue a
- They use the same cluster name. See :doc:`configure`
- They share the same ``SECRET_KEY`` for Django.
.. _multiple-queues
Multiple Queues
-----------------
You can have multiple queues in one Django site, and use multiple cluster to work on each queue.
Different queues are identified by different queue names which are also cluster names.
To run an alternate cluster, e.g. to work on the 'long' queue, start your cluster with command::
# On Linux
$ Q_CLUSTER_NAME=long python manage.py qcluster
# On Windows
$ python manage.py qcluster --name long
You can set different Q_CLUSTER options for alternative clusters, such as 'timeout', 'queue_limit'
and any other options which are valid in :doc:`configure`. See :ref:`alt-clusters`.
.. note::
To use multiple queue, use the keyword argument `cluster` in async_task() and schedule():
* if `cluster` is not set (the default), async_task() and schedule() will be handled by the default cluster;
* if `cluster` is set, only clusters with matching cluster name will run the task or do the schedule.
Using a Procfile
----------------
If you host on `Heroku <https://heroku.com>`__ or you are using `Honcho <https://github.com/nickstenning/honcho>`__ you can start the cluster from a :file:`Procfile` with an entry like this::

View File

@@ -73,9 +73,9 @@ author = "Ilan Steemers, Stan Triepels"
# built documents.
#
# The short X.Y version.
version = "1.4"
version = "1.5"
# The full version, including alpha/beta/rc tags.
release = "1.4.6"
release = "1.5.4"
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.

View File

@@ -20,7 +20,18 @@ Configuration is handled via the ``Q_CLUSTER`` dictionary in your :file:`setting
'redis': {
'host': '127.0.0.1',
'port': 6379,
'db': 0, }
'db': 0, },
'ALT_CLUSTERS': {
'long': {
'timeout': 3000,
'retry': 3600,
'max_attempts': 2,
},
'short': {
'timeout': 10,
'max_attempts': 1,
},
}
}
All configuration settings are optional:
@@ -70,6 +81,13 @@ Set this to something that makes sense for your project. Can be overridden for i
See :ref:`retry` for details how to set values for timeout and retry.
.. _time_zone:
time_zone
~~~~~~~
The timezone that is used for task scheduling. Use this if you are having issue with DST. The scheduler uses UTC to calculate the next date and will therefore ignore any DST changes. This will cause 1 hour or 0.5 hour changes in the schedule when time is moved one hour ahead or back. Defaults to `settings.TIME_ZONE` if `USE_TZ` is enabled.
.. _ack_failures:
ack_failures
@@ -310,16 +328,6 @@ Using the Django ORM backend will also enable the Queued Tasks table in the Admi
If you need better performance , you should consider using a different database backend than the main project.
Set ``orm`` to the name of that database connection and make sure you run migrations on it using the ``--database`` option.
When using the Django database as a message broker, you can set the ``has_replica`` boolean keyword to ensure Django-Q2 works properly letting a `Database Router <https://docs.djangoproject.com/en/3.2/topics/db/multi-db/>`__. ::
# example ORM broker connection with replica database
Q_CLUSTER = {
...
'orm': 'default',
'has_replica': True
}
.. _mongo_configuration:
mongo
@@ -462,6 +470,24 @@ As a rule of thumb; cpu_affinity 1 favors repetitive short running tasks, while
*Psutil does not support cpu affinity on OS X at this time.*
.. _alt-clusters:
ALT_CLUSTERS
~~~~~~~~~~~~
For multiple clusters working on multiple queues to run in one Django site.
ALT_CLUSTERS should be a dict with cluster_name as its key, and the value is the configuration for the cluster
with the key as its name. The configuration items are consistent with Q_CLUSTER,
except for a few items such as name/cluster_name/ALT_CLUSTER, which are not available of course.
See :ref:`multiple-queues`.
.. note::
For a cluster, if its name is in ALT_CLUSTERS, the config item in ALT_CLUSTER will override
the same config item in the Q_CLUSTER root. Other config items in Q_CLUSTER root remain in effect for this cluster.
.. py:module:: django_q
.. rubric:: Footnotes

View File

@@ -27,7 +27,7 @@ Features
- Rollbar and Sentry support
Django Q2 is tested with: Python 3.8, 3.9 and 3.10, 3.11, Django 3.2.x and 4.1.x
Django Q2 is tested with: Python 3.8, 3.9, 3.10 and 3.11. Works with Django 3.2.x, 4.1.x and 4.2.x
Currently available in English, German and French.

View File

@@ -32,7 +32,7 @@ Django Q2 is tested for Python 3.8, 3.9, 3.10 and 3.11
- `Django <https://www.djangoproject.com>`__
Django Q2 aims to use as much of Django's standard offerings as possible.
The code is tested against Django versions `3.2.x` and `4.1.x`.
The code is tested against Django versions `3.2.x`, `4.1.x` and`4.2.x`.
- `Django-picklefield <https://github.com/gintas/django-picklefield>`__
@@ -55,6 +55,10 @@ Optional
$ pip install psutil
- `setproctitle <https://github.com/dvarrazzo/py-setproctitle>`__ python module to customize the process title by Daniele Varrazzo', is an optional requirement used to set informative process titles::
$ pip install setproctitle
- `Hiredis <https://github.com/redis/hiredis>`__ parser. This C library maintained by the core Redis team is faster than the standard PythonParser during high loads::
$ pip install hiredis
@@ -136,7 +140,7 @@ You can reference the `requirements <https://github.com/GDay/django-q2/blob/mast
Django
~~~~~~
We strive to be compatible with last two major version of Django.
At the moment this means we support the 3.2.x and 4.1.x releases.
At the moment this means we support the 3.2.x, 4.1.x and 4.2.x releases.
Since we are now no longer supporting Python 2, we can also not support older versions of Django that do not support Python >= 3.6
For this you can always use older releases, but they are no longer maintained.

View File

@@ -69,6 +69,10 @@ You can change this by setting the :ref:`catch_up` configuration setting to ``Fa
The scheduler will then skip execution of scheduled events in the past.
Instead those tasks will run once when the cluster starts again and the scheduler will find the next available slot in the future according to original schedule parameters.
When :ref:`catch_up` is to ``True`` it may be useful for the task to know what was the date and time it was originally intended to run at.
To achieve this, pass an identifier name to parameter `intended_date_kwarg` when creating the schedule. The intended datetime will then be passed - in isoformat string - as
a kwarg with that identifier name to the task that has been created.
Management Commands
-------------------
@@ -123,9 +127,10 @@ Reference
:param int repeats: Number of times to repeat schedule. -1=Always, 0=Never, n =n.
:param datetime next_run: Next or first scheduled execution datetime.
:param str cluster: optional cluster name. Task will be executed only on a cluster with a matching :ref:`name`.
:param str intended_date_kwarg: optional identifier to pass intended schedule date.
:param dict q_options: options passed to async_task for this schedule
:param kwargs: optional keyword arguments for the scheduled function.
.. note::
q_options does not accept the 'broker' key with a broker instance but accepts a 'broker_name' key instead. This can be used to specify the broker connection name to assign the task. If a broker with the specified name does not exist or is not running at the moment of placing the task in queue it fallbacks to the random broker/queue that handled the schedule.
@@ -183,9 +188,13 @@ Reference
When set to -1, this will keep counting down.
.. py:attribute:: cluster
Task will be executed only on a cluster with a matching :ref:`name`.
.. py:attribute:: intended_date_kwarg
Name of kwarg to pass intended schedule date.
.. py:attribute:: next_run
Datetime of the next scheduled execution.

View File

@@ -13,6 +13,12 @@ Before enqueuing a task
The ``django_q.signals.pre_enqueue`` signal is emitted before a task is
enqueued. The task dictionary is given as the ``task`` argument.
After spawning a worker process
"""""""""""""""""""""""""""""""
The ``django_q.signals.post_spawn`` signal is emitted after a worker process has
spawned. The process name is given as the ``proc_name`` argument (string).
Before executing a task
"""""""""""""""""""""""
@@ -37,7 +43,7 @@ Connecting to a Django Q2 signal is done the same as any other Django
signal::
from django.dispatch import receiver
from django_q.signals import pre_enqueue, pre_execute, post_execute
from django_q.signals import pre_enqueue, pre_execute, post_execute, post_spawn
@receiver(pre_enqueue)
def my_pre_enqueue_callback(sender, task, **kwargs):
@@ -51,4 +57,8 @@ signal::
def my_post_execute_callback(sender, task, **kwargs):
print(f"Task {task['name']} was executed with result {task['result']}")
@receiver(post_spawn)
def my_post_spawn_callback(sender, proc_name, **kwargs):
print(f"Process {proc_name} has spawned")

1635
poetry.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
[tool.poetry]
name = "django-q2"
version = "1.4.6"
version = "1.5.4"
packages = [
{ include = "django_q" },
]
@@ -44,7 +44,7 @@ include = ['CHANGELOG.md']
[tool.poetry.dependencies]
python = ">=3.8.14, <4"
python = ">=3.8, <4"
django = ">=3.2"
django-picklefield = "^3.1"
@@ -59,6 +59,7 @@ croniter = { version = "^1.3.7", optional = true }
django-q-rollbar = {version = ">=0.1", optional = true}
django-q-sentry = {version = ">=0.1", optional = true}
redis = {version = "^4.3.4", optional = true}
setproctitle = {version = "^1.3.2", optional = true}
[tool.poetry.dev-dependencies]
@@ -70,9 +71,9 @@ black = "^22.10.0"
isort = {extras = ["requirements_deprecated_finder"], version = "^5.10.1"}
[tool.poetry.extras]
requires = ["poetry_core>=1.0.0"]
requires = ["poetry_core"]
build-backend = ["poetry.core.masonry.api"]
testing = ["django-redis", "croniter", "hiredis", "psutil", "iron-mq", "boto3", "pymongo", "blessed", "redis"]
testing = ["django-redis", "croniter", "hiredis", "psutil", "iron-mq", "boto3", "pymongo", "blessed", "redis", "setproctitle"]
rollbar = ["django-q-rollbar"]
sentry = ["django-q-sentry"]