Commit Graph
75 Commits
Author SHA1 Message Date
Stan TriepelsandWesley Ellis 00fe611b86 Use logger.hasHandlers() to setup fallback logging (#13)
Instead of checking if the django-q logger has any handlers directly,
use the hasHandlers [1] method, which will check if any parent loggers
have handlers as well.

This fixes a bug where django-q configures it's own logging even though
the root logger has a handler set

 1. https://docs.python.org/3/library/logging.html#logging.Logger.hasHandlers

Co-authored-by: Wesley Ellis <wellis@benchsci.com>
2022-10-15 17:39:51 +02:00
Stan TriepelsandNoortheen Raja 9ace854290 Save limit per group/func/name (#12)
* feat: option to save tasks per group/func/name

* test: update tests for save limit

* fix: convert func when save-limit checking

when passed as function it needs to be converted to work

* Add warning if option is not valid

Co-authored-by: Noortheen Raja <jnoortheen@gmail.com>
2022-10-15 01:56:41 +02:00
Stan TriepelsandMarc Sabatier e3f959c258 Admin improvements (#11)
* Add the group column to OrmQ admin page

* Add group field to tasks list and search_field

* Add search by name to Schedule admin page

* Limit succesful task history per schedule with SAVE_LIMIT_GROUP config

* Improve ScheduleAdmin list view performance

Co-authored-by: Marc Sabatier <marc@sabatier.online>
2022-10-14 17:13:48 +02:00
Ilan Steemers 15155c7a99 Build improvements (#569)
* Black linting

* Adding Black to dev dependencies and upping minimal python to 3.6.2 for compatibility

* Updating packages

* Removing pip-tools input and exporting requirements with poetry

* Deleting old setup files and test runner

* Trying 1.3.7

* Looser extras requirements to prevent conflicts

* Sorted imports with isort

* Added iSort to dev dependencies

* Fixes localtime for naive setups
2021-05-30 17:10:07 +02:00
Alexandre Xavier 71d4b726ea Feature/improves multiple databases support (#561)
* feat: adds docker local files to ignore

* feat: adds replica key to django q settings

* feat: adds multiple database routers testing utility

* feat: ensures scheduler works with multiple databases and replicas

* feat: adds docker image to support containerized development

* refactor: removes unnecessary import

* style: improves import styling

* docs: adds replica setting to configure documentation
2021-05-23 15:10:31 +02:00
RubenandIlan Steemers 7892aef70a Add a warning for misconfiguration. (#509)
* Add a warning  for misconfiguration.

As specified in the docs, by default the settings result in a configuration where slow tasks that run for more than 60s cause repeated running of tasks, which has caused a few issues. This warning doesn't change the defaults, which would be a marge larger change, but instead provides a, hopefully, helpful message to the user.

* fixed missing bracket

* Update conf.py

fixed spelling

* Update conf.py

Made bool work for python 3.6+

* Fixing typo

Co-authored-by: Ilan Steemers <koed00@gmail.com>
2021-02-26 13:54:56 +01:00
Tim O'Meara 655f0aadb5 rename config var to max_retries, remove attempt_count form admin , add admin ui customization to docs 2020-08-13 11:58:00 -05:00
Tim O'Meara 3c3395f24b add attempt_count to task 2020-08-11 10:16:31 -05:00
Ilan Steemers c94f2cb816 Adds psutil support for max_rss on other platforms 2020-07-05 18:18:43 +02:00
Ilan Steemers b16878f36f Allow for platforms not supporting resource module 2020-07-05 17:45:07 +02:00
Ilan Steemers 83b8b7c0d7 Adds max_rss memory limit for recycle 2020-07-05 17:04:16 +02:00
Ilan Steemers 6e95dc2d9b Hints and lints 2020-06-15 16:02:14 +02:00
Bastien Vallet 68d95a2924 [cleanup] Remove python2 artifacts 2020-06-10 11:48:31 +02:00
Ilan 0bca3cbd7a Replacing format with f-strings 2020-02-17 16:59:12 +01:00
Kevin Lord 0d9ebf94ee Fix startup error in AWS Lambda 2020-01-20 22:39:40 -06:00
Jake Bell dc65f5e1de Replacing ugettext_ functions with gettext_ for Django 3 2020-01-07 15:57:37 -06:00
Ilan Steemers b5a01a9e93 Removes old rollbar support - relies on the pluggable error reporters now 2018-08-10 13:03:58 +02:00
Balletie 03abbc960f Add option for acknowledging failed tasks (globally and per-task)
If a task fails with an exception, it is retried until it
succeeds. This is contrary to what is said in the documentation: under
the "Architecture" section, heading "Broker" it says that even when a
task errors, it's still considered a successful delivery. Failed tasks
never get acknowledged however, thereby being retried after the
timeout period. See also issues #238 and #194.

This patch adds an option to acknowledge failures, thereby closing
issue #238. Issue #194 would require some more work. The default of
this option is set to `False`, thereby maintaining backwards
compatibility.
2018-03-09 22:09:46 +01:00
Ronald van Zon dd4f02f122 Add custom SharedCounter / Queue
This will not fix anything related to Django 2.0 but will allow MAC
users to run Django-Q allowing tests to be run locally.
2017-12-13 14:36:12 +01:00
Daniel Welch c6466868d8 fix usage of iter_entry_points 2017-12-07 18:30:54 -05:00
Daniel Welch 4530b05d86 putting back support for backwards compatibility 2017-10-06 13:26:43 -04:00
Daniel Welch cd038f6f61 remove old rollbar stuff from config 2017-10-03 17:08:26 -04:00
Daniel Welch c82ef6a177 fixing export from conf 2017-03-30 22:12:31 -04:00
Daniel Welch 9e97d48012 registering entry points for sentry and rollbar plugins, maintained in seperate repos. start of a generic error handling implementation 2017-03-30 21:51:57 -04:00
Yann Pomarede aa667b40d4 daemonize_workers option 2016-12-05 14:55:42 +01:00
ilan 6734f5b96c Removes value check for guard cycle 2016-07-21 16:41:30 +02:00
Vladimir Rychkov cadbc9f220 exception raised if guard_cycle configured not in range (0, 60) 2016-07-13 09:44:26 +01:00
Vladimir Rychkov 1a1385dfba Guard loop sleep made configurable 2016-07-12 13:08:17 +01:00
Ilan Steemers c17468ef86 docs: Adds Custom broker class documentation
Also changes some of the variable and settings names, to make them less ambiguous.
2016-01-27 12:06:49 +01:00
Ilan Steemers 8298f2319f Fixes issue with rollbar configuration 2016-01-27 10:32:48 +01:00
Ilan Steemers af3b1da08f Makes DB polling time configurable via poll setting 2016-01-26 22:29:00 +01:00
Ilan Steemers 91bb1c69c7 Fixes double log output 2016-01-08 13:37:58 +01:00
Ilan Steemers d29c861d09 Adds Rollbar support for exceptions
Also formats exceptions in case they are unprintable
2016-01-07 18:50:55 +01:00
Ilan Steemers 33282cb2e5 cached result backend
first version of a result backend using django's cache framework
2015-10-01 14:58:28 +02:00
Ilan Steemers b8bb186471 check for default database on mongo connection 2015-09-26 13:01:15 +02:00
Ilan Steemers 826178c463 Adds a MongoDB broker
* adds mongo broker tests
* adds mongo brokers docs

* broker info is now cached when it needs to connect to a server, to reduce traffic.
2015-09-25 20:03:39 +02:00
Ilan Steemers 2bffcf554b Django ORM broker 2015-09-14 13:24:43 +02:00
Ilan Steemers 8543c5b328 Adds Fastack option to Disque broker 2015-09-11 17:58:16 +02:00
Ilan Steemers d7ce945f03 Amazon SQS broker 2015-09-09 18:10:03 +02:00
Ilan Steemers 83548dcfbb Makes bulk a general setting 2015-09-08 13:42:15 +02:00
Ilan Steemers 0ffae4519b Adds bulk get to ironmq 2015-09-08 13:37:33 +02:00
Ilan Steemers b56cc9da8f IronMQ broker 2015-09-08 11:20:31 +02:00
Ilan Steemers 73d15c3608 docs: minor updates 2015-09-06 16:54:20 +02:00
Ilan Steemers b201979eae defaults queue_limit to workers squared. 2015-09-06 15:45:17 +02:00
Ilan Steemers 3f75e7fa05 removed SQS and IronMQ brokers for now
Both brokers are not atomic and cause all kinds of problems during testing. Needs more time.
Meanwhile I want to release the pluggable broker backend.
2015-09-04 14:49:17 +02:00
Ilan Steemers c626c3137c Defaults RETRY to 60 2015-09-03 14:40:32 +02:00
Ilan Steemers b1f2143a3d uses disque_nodes convention 2015-09-03 12:55:49 +02:00
Ilan Steemers 8ce441bed5 Adds more brokers
* added IronMQ
* added Aws SQS
* removed SafeRedis.
 During tests the 'safe' part wasn't consistent enough
2015-09-02 19:04:03 +02:00
Ilan Steemers 4f39062785 Adds django cache as monitor cache
Not the best solution yet, but it solves monitoring problems with brokers that don't have a pattern getter for cluster stats.
2015-08-30 18:28:26 +02:00
Ilan Steemers d3283571a6 First draft of a Disque broker
TODO monitoring doesn't work yet
2015-08-30 14:33:35 +02:00