Commit Graph
81 Commits
Author SHA1 Message Date
Stan Triepels e78e473be3 Fix: handling exceptions inside job function (#51) 2022-12-21 01:44:59 +01:00
Stan Triepels 3c0c32758d Chore: Flake8, isort and Black (#40) 2022-11-16 01:45:40 +01:00
Stan Triepels 0aac4e7b0d Fix unclear error when function is not called correctly (#19) 2022-10-20 17:24:17 +02:00
Stan Triepels c59f036158 Remove arrow dependency, update packages, set up testing with compose (#17) 2022-10-19 02:01:16 +02:00
Stan TriepelsandNoortheen Raja 9ace854290 Save limit per group/func/name (#12)
* feat: option to save tasks per group/func/name

* test: update tests for save limit

* fix: convert func when save-limit checking

when passed as function it needs to be converted to work

* Add warning if option is not valid

Co-authored-by: Noortheen Raja <jnoortheen@gmail.com>
2022-10-15 01:56:41 +02:00
Ilan Steemers f8cd136668 Post execute signal and tests (#580)
* adds post_execute signal and tests

* Updates signal docts

* Fixes typo in docs
2021-06-20 15:43:08 +02:00
Ilan Steemers 15155c7a99 Build improvements (#569)
* Black linting

* Adding Black to dev dependencies and upping minimal python to 3.6.2 for compatibility

* Updating packages

* Removing pip-tools input and exporting requirements with poetry

* Deleting old setup files and test runner

* Trying 1.3.7

* Looser extras requirements to prevent conflicts

* Sorted imports with isort

* Added iSort to dev dependencies

* Fixes localtime for naive setups
2021-05-30 17:10:07 +02:00
Ilan Steemers 7d03536e89 Migrate to Github Action CI (#507)
* Some small refactors

* Initial test script

* Fixes django matrix

* Trying to set up Disque

* Disque as a docker service

* lowercase action name

* Remove Travis

* Replaces Travis badge with Github actions badge

* Show django version in step

* Typo
2021-02-14 18:59:04 +01:00
alx-sdv 28197dc4fa Model.__unicode__() has no effect 2020-10-05 12:30:11 +03:00
Tim O'Meara 655f0aadb5 rename config var to max_retries, remove attempt_count form admin , add admin ui customization to docs 2020-08-13 11:58:00 -05:00
Tim O'Meara f26de929d6 adds tests 2020-08-11 11:48:23 -05:00
Ilan Steemers 0095a93b45 Adds max rss test 2020-07-08 22:52:07 +02:00
Ilan Steemers 572f810651 Adds max rss test 2020-07-08 22:47:35 +02:00
Ilan Steemers e6f82c5158 test model unicode 2020-06-30 22:00:31 +02:00
Bastien Vallet 3569d2ea8c [cleanup] Fix some linting issues 2020-06-10 12:00:55 +02:00
Bastien Vallet 68d95a2924 [cleanup] Remove python2 artifacts 2020-06-10 11:48:31 +02:00
Ryan Branche d7d0b0dfb4 Add unit test for sync=True change.
When sync=True, re-raise exceptions from the worker.
2020-02-28 14:47:28 -08:00
Jason McVetta 14a602ee8b use both pid and cluster_id 2020-02-12 18:23:47 +07:00
Jason McVetta d22c916bfc add cluster_id to Sentinel 2020-02-12 17:53:52 +07:00
Janne Rönkkö 0e2df88d92 Fix async_task timeout parameter handling when cluster timeout is set to None (the default)
The cluster timeout configuration default value None is documented to mean
tasks never timeout out. Also the documentation states that the timeout
can be overridden for individual tasks.

With the old implementation the timeout parameter given to async_task was
not honored if the cluster timeout was set to None.

Fixes: #335
2019-01-27 19:09:22 +02:00
Janne Rönkkö 2650659da9 Allow timeout unit tests to fail properly
The old implementation used count_forever test task that never finishes.
If the timeout implementation does not work in the test, the test never
end and you was required to kill the test runner.
2019-01-27 15:35:17 +02:00
Janne Rönkkö d09e2658be Add test for task finishes before timeout 2019-01-27 15:35:17 +02:00
Janne Rönkkö 888324cf48 Reduce code duplication in timeout unit tests 2019-01-26 20:44:31 +02:00
Ilan Steemers 7d5c9f8b46 removes xrange compat 2018-08-05 13:39:04 +02:00
Ilan Steemers acebdaa850 Renames async/enqueue to async_task. 2018-08-01 15:20:34 +02:00
Pierre-Elliott Bécue ae75a84f7f Replaces async occurrences with alternatives
* async is now a reserved word in python3.7
 * Rename async function to enqueue
 * Rename all async_ functions to enqueue_
 * Rename Async class to AsyncTask
 * Updates the docs.
2018-07-07 21:33:43 +02:00
Balletie 03abbc960f Add option for acknowledging failed tasks (globally and per-task)
If a task fails with an exception, it is retried until it
succeeds. This is contrary to what is said in the documentation: under
the "Architecture" section, heading "Broker" it says that even when a
task errors, it's still considered a successful delivery. Failed tasks
never get acknowledged however, thereby being retried after the
timeout period. See also issues #238 and #194.

This patch adds an option to acknowledge failures, thereby closing
issue #238. Issue #194 would require some more work. The default of
this option is set to `False`, thereby maintaining backwards
compatibility.
2018-03-09 22:09:46 +01:00
Ronald van Zon dd4f02f122 Add custom SharedCounter / Queue
This will not fix anything related to Django 2.0 but will allow MAC
users to run Django-Q allowing tests to be run locally.
2017-12-13 14:36:12 +01:00
ilan 8a7567bc77 Replaces some relative imports 2017-10-10 12:13:33 +02:00
Benjamin Bach 5ea55e3718 Import python3 version of range 2017-07-09 21:44:25 +02:00
Harm Geerts cf37efb1eb Replace global Conf mangling with monkeypatch
The test suite, especially for the brokers, was heavily dependant on the
success of the tests preceeding it to pass. This commit should eliminate
that dependency and allow tests to fail without affecting others.
It also removes the need to cleanup globals manually after a test.
2016-11-29 18:08:52 +01:00
Ilan Steemers 9741efdc5e Adds test for task result update 2016-01-24 21:30:42 +01:00
Ilan Steemers 8b27cdae17 Removes duplicate test 2016-01-09 13:42:13 +01:00
Ilan Steemers 9243731a9c Updates test for new wait and count options 2015-10-02 20:03:38 +02:00
Ilan Steemers 826178c463 Adds a MongoDB broker
* adds mongo broker tests
* adds mongo brokers docs

* broker info is now cached when it needs to connect to a server, to reduce traffic.
2015-09-25 20:03:39 +02:00
Ilan Steemers ef6a60478a Merge pull request #74 from Koed00/dev
Removes root imports for Django 1.9
2015-09-25 12:35:10 +02:00
Ilan Steemers b3381d6dae removes root imports for django 1.9
typo in test
2015-09-24 15:47:40 +02:00
Ilan Steemers cef48da789 removes root imports for django 1.9
Updated docs to promote submodule imports
2015-09-24 15:41:32 +02:00
Ilan Steemers 985fbb31cc tests cluster stop when already stopped 2015-09-24 11:21:28 +02:00
Ilan Steemers a493f9cc78 Adds wait option to result() and fetch() 2015-09-18 10:57:09 +02:00
Ilan Steemers 061775b3bb Adds group commands to task instances
* adds task.group_result()
* adds task.group_count()
* adds task.group_delete()
2015-09-16 15:47:39 +02:00
Ilan Steemers 2bffcf554b Django ORM broker 2015-09-14 13:24:43 +02:00
Ilan Steemers e4a5951f12 Resets SQS conf in case of test failure 2015-09-09 18:59:24 +02:00
Ilan Steemers 8eb1431413 Resets broker configuration in case a third party broker crashes 2015-09-08 15:50:32 +02:00
Ilan Steemers c946a16b99 changes py 2.7 import for ironmq 2015-09-08 15:38:00 +02:00
Ilan Steemers f4f8af1382 Fixes backward compatibility problems with django-picklefield 2015-09-07 20:24:21 +02:00
Ilan Steemers 3f75e7fa05 removed SQS and IronMQ brokers for now
Both brokers are not atomic and cause all kinds of problems during testing. Needs more time.
Meanwhile I want to release the pluggable broker backend.
2015-09-04 14:49:17 +02:00
Ilan Steemers 0740c90148 adds pluggable brokers
* added redis broker
 * added django_redis broker
 * added task acknowledgement
2015-08-29 18:54:07 +02:00
Ilan Steemers 439c8a2f77 Moves Status to it's own file
This way only qmonitor and qinfo need Curses and you can still run the cluster without it.
2015-08-26 13:11:58 +02:00
Ilan Steemers 3e92fddcdd #50 removes redis connection argument from pusher process 2015-08-24 10:08:43 +02:00