* AttributeError when start_event is None, and guard process faster stop
This commit has two related fixes that come into play when frequently
shutting down and restarting the cluster.
If a SIGINT or SIGTERM signal is received while the main process is
waiting for the sentinel process to set start_event, the signal
handler sets start_event to None, and the loop that polls start_event
will raise an unhandled AttributeError attempting to check
start_event.is_set().
The second issue is that the guard process sleeps for the guard cycle
interval in each iteration before checking the stop event, preventing
the guard cycle from terminating until the sleep completes. The
cluster can be made more responsive to a shutdown request by using
the event's wait(cycle) method rather than time.sleep(cycle) so the
process wakes immediately when the event is set.
The commit also fixes the case where the guard process sleeps for an
extra cycle when the cycle counter happens to have been reset to zero
on the loop iterations when the scheduler is called.
This fix helps in our environment where we are frequently stopping
and starting the cluster and where we have increased the guard
cycle setting to several seconds.
* Add sentinel premature death check while waiting for sentinel to start.
Suggested by github copilot [here](https://github.com/django-q2/django-q2/pull/305/changes#r3143964420)
* Add test case test_cluster_early_stop
Add a test case to validate stopping the cluster before the
sentinel has set the cluster's start_event.
The test deterministically triggers the AttributeError when
start_event is None (without the PR's fix in cluster.py).
As requested by copilot: https://github.com/django-q2/django-q2/pull/305#discussion_r3143964399
* Add test test_cluster_stop_responsive
Ensure that stopping the cluster is responsive and does not wait for a full
GUARD_CYCLE to stop the cluster.
As requested by copilot: https://github.com/django-q2/django-q2/pull/305/changes#r3143964414
* Address copilot review comments.
1. Do not raise RuntimeError if sentinel early exit was caused by
SIGINT/SIGTERM stopping the cluster while waiting for the
sentinel to start.
Addresses https://github.com/django-q2/django-q2/pull/305#discussion_r3298618046
2. Ensure that the monkeypatched broker instance remains pickleable
so the test_cluster_early_stop test will also work on platforms
that use the spawn Process start method.
Addresses https://github.com/django-q2/django-q2/pull/305#discussion_r3298618086
3. Do not allow the test_cluster_early_stop test to hang if the
sentinel process or main test process terminates unexpectedly
without setting the sentinel_event or test_event.
Addresses https://github.com/django-q2/django-q2/pull/305#discussion_r3298618102
Prune stale entries from the master list on new cluster registration
and only write the list when membership changes (with timeout=None).
Before this fix, set_stat was append-only — dead cluster keys accumulated
forever because the only cleanup path (get_stats) runs only from the
monitor UIs. On DatabaseCache backends this rewrote the growing pickled
list to Postgres on every heartbeat, producing measurable egress.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Move logic that resubmits a task and then deletes the original into an action function that can be used for both failure and success resubmission.
* Use the resubmit_task action in the FailAdmin.
* Add the resubmit_task action to the TaskAdmin class.
* Update resubmit failure test to reflect using the new resubmit action.
* Add a test for resubmitting successful tasks.
---------
Co-authored-by: Scott Pashley
* Support for multi-queue, multi-cluster configuration. API changes include:
* Adding `cluster` to async_task() parameters and Task model
* Adding argument --name to qcluster command
* Necessary adjustments to Conf and Broker classes
* Some admin improvements
* Add settings.Q_CLUSTER['ALT_CLUSTERS']: q_cluster config overrides for alternative clusters;
Add Conf.CLUSTER_NAME: separate usage from Conf.PREFIX;
QueueAdmin/OrmQ detail page enhanced: now displaying args/kwargs/q_options instead of encrypted payload.
* if `cluster` argument is not set (the default), async_task() and schedule() will be handled by the default cluster; Documentation update.
* Text cleanup
* Documentation update.
* Fix TIMEOUT setting in Windows for non-default cluster
---------
Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
* feat: option to save tasks per group/func/name
* test: update tests for save limit
* fix: convert func when save-limit checking
when passed as function it needs to be converted to work
* Add warning if option is not valid
Co-authored-by: Noortheen Raja <jnoortheen@gmail.com>
* Black linting
* Adding Black to dev dependencies and upping minimal python to 3.6.2 for compatibility
* Updating packages
* Removing pip-tools input and exporting requirements with poetry
* Deleting old setup files and test runner
* Trying 1.3.7
* Looser extras requirements to prevent conflicts
* Sorted imports with isort
* Added iSort to dev dependencies
* Fixes localtime for naive setups
* Allows schedule by cluster
* Update docs
* Add example in doc
* Fixes cluster field must be blank
* Adds cluster arg to schedule func
* Adds cluster field in list_display/filter for admin
* Uses Q filter to fetch schedule
* Fixes ref in doc
* Adds tests
* added long pooling support for sqs broker
* changed the parameter name
* added long polling support ,
introduce new parameter in broker sqs block
parameter for long polling is receive_message_wait_time_seconds
eg :
Q_CLUSTER = {
'name': 'test-queue',
'sqs': {
'aws_region': AWS_S3_REGION_NAME,
'aws_access_key_id': AWS_ACCESS_KEY_ID,
'aws_secret_access_key': AWS_SECRET_ACCESS_KEY,
'receive_message_wait_time_seconds':20
}
}
* added receive_message_wait_time_seconds parameter in test_broker.py
Co-authored-by: Javed Gouri <you@example.com>
* Some small refactors
* Initial test script
* Fixes django matrix
* Trying to set up Disque
* Disque as a docker service
* lowercase action name
* Remove Travis
* Replaces Travis badge with Github actions badge
* Show django version in step
* Typo
URLs were linking to a now unsupported version of Django. Django v2.2 is a LTS version, and also matches the documentation intersphinx_mapping in docs/conf.py.
David Macario