• 欢迎访问蜷缩的蜗牛博客 蜷缩的蜗牛
  • 微信搜索: 蜷缩的蜗牛 | 联系站长 kbsonlong@qq.com
  • 如果您觉得本站非常有看点,那么赶紧使用Ctrl+D 收藏吧
  • 本站已开通微信小程序,可以扫描底部二维码关注,或者搜索alongparty!!

Django2集成celery4执行异步任务、定时任务


1 环境依赖

Python3.7

#依赖库
Django==2.1.7
celery==4.1.0
redis==3.2.1
#Windows 安装
eventlet (加入协程支持)

2 创建 django 项目

# django-admin startproject testprj
testprj/
├── manage.py
└── testprj
    ├── __init__.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py

1 directory, 5 files


3 创建 celery_tasks 模块

# cd testprj
# mkdir celery_tasks
tree celery_tasks/
celery_tasks/
├── celery.py
├── __init__.py
└── tasks.py

0 directories, 3 files

3.1 celery.py

# -*- coding: utf-8 -*-
# @Time    : 2019-04-30 11:55
# @Author  : kbsonlong
# @Email   : kbsonlong@gmail.com
# @Blog    : www.alongparty.cn
# @File    : celery.py
# @Software: PyCharm

from __future__ import absolute_import, unicode_literals
from celery import Celery

BROKER_URL = 'redis://localhost:6379/1'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/2'
app = Celery('testprj',
             broker=BROKER_URL,
             backend=CELERY_RESULT_BACKEND,
             include=['celery_tasks.tasks'])

# Optional configuration, see the application user guide.
app.conf.update(
    result_expires=3600,
    task_serializer='json',
    accept_content=['json'],  # Ignore other content
    result_serializer='json',
    timezone='Asia/Shanghai',
    enable_utc=False,
)

if __name__ == '__main__':
    app.start()

3.2 tasks.py

# -*- coding: utf-8 -*-
# @Time    : 2019-04-30 12:00
# @Author  : kbsonlong
# @Email   : kbsonlong@gmail.com
# @Blog    : www.alongparty.cn
# @File    : tasks.py
# @Software: PyCharm

from __future__ import absolute_import, unicode_literals
from celery_tasks.celery import app


@app.task
def add(x, y):
    return x + y


@app.task
def mul(x, y):
    return x * y


@app.task
def xsum(numbers):
    return sum(numbers)

4 Django 项目配置 celery

vim testprj/testprj/init.py

from celery_tasks.celery import app as celery_app
__all__ = ['celery_app']

5 启动 celery work

# celery -A proj worker -l info 
-------------- celery@LAPTOP-Q26TVKSR v4.3.0 (rhubarb)
---- **** -----
--- * ***  * -- Windows-10-10.0.17134-SP0 2019-04-30 15:07:19
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         ops:0x17d108a5da0
- ** ---------- .> transport:   redis://localhost:6379/1
- ** ---------- .> results:     redis://localhost:6379/2
- *** --- * --- .> concurrency: 8 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . celery_tasks.tasks.add
  . celery_tasks.tasks.mul
  . celery_tasks.tasks.xsum

[2019-04-30 15:07:24,198: INFO/MainProcess] Connected to redis://localhost:6379/1
[2019-04-30 15:07:32,616: INFO/MainProcess] mingle: searching for neighbors
[2019-04-30 15:07:46,658: INFO/MainProcess] mingle: all alone
[2019-04-30 15:07:59,528: INFO/MainProcess] celery@LAPTOP-Q26TVKSR ready.
[2019-04-30 15:08:05,911: INFO/MainProcess] pidbox: Connected to redis://localhost:6379/1.
  • 从启动信息上可以看到有两部分内容,一部分为 celery 的配置信息,一部分为 tasks;
  • 从[tasks] 选项中可以看到刚才在 tasks.py 定义的 task 都加载进来了。

6 如何调用定义的任务

6.1 外部文件加载 tasks,如 views

from celery_tasks.tasks import add,mul,xsum

6.2 调用任务(task)

6.2.1 调用直接返回结果
add.delay(2,2)
6.2.2 异步调用
add.apply_async((2, 2))
6.2.3 异步调用,并在多久之后执行
res=add.apply_async((2, 2), queue='celery', countdown=10)
print(res.id,res.status)
  • 任务将被发送到一个名为的队列 celery,任务将在发送消息后 10 秒执行
  • 打印出任务 id 和状态
9f3a4f5b-c143-4fac-b26f-815df7ec3678 PENDING
  • 登录 redis,查看执行结果
{
status: "SUCCESS",
result: 4,
traceback: null,
children: [
],
task_id: "9f3a4f5b-c143-4fac-b26f-815df7ec3678",
date_done: "2019-04-30T07:52:32.543102"
}

运行 celery4.0 如果出现如下异常

  File "celery/backends/redis.py", line 22
    from . import async, base
                      ^
SyntaxError: invalid syntax
解决方案: pip install https://github.com/celery/celery/tarball/master
https://github.com/celery/celery/pull/4852/commits/d737dec3c943632f21f73a2235409c29e3fe63e3

参考资料

celery 官网


蜷缩的蜗牛 , 版权所有丨如未注明 , 均为原创丨 转载请注明Django2 集成 celery4 执行异步任务、定时任务
喜欢 (0)
[]
分享 (0)

Django2集成celery4执行异步任务、定时任务


1 环境依赖

Python3.7

#依赖库
Django==2.1.7
celery==4.1.0
redis==3.2.1
#Windows 安装
eventlet (加入协程支持)

2 创建 django 项目

# django-admin startproject testprj
testprj/
├── manage.py
└── testprj
    ├── __init__.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py

1 directory, 5 files


3 创建 celery_tasks 模块

# cd testprj
# mkdir celery_tasks
tree celery_tasks/
celery_tasks/
├── celery.py
├── __init__.py
└── tasks.py

0 directories, 3 files

3.1 celery.py

# -*- coding: utf-8 -*-
# @Time    : 2019-04-30 11:55
# @Author  : kbsonlong
# @Email   : kbsonlong@gmail.com
# @Blog    : www.alongparty.cn
# @File    : celery.py
# @Software: PyCharm

from __future__ import absolute_import, unicode_literals
from celery import Celery

BROKER_URL = 'redis://localhost:6379/1'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/2'
app = Celery('testprj',
             broker=BROKER_URL,
             backend=CELERY_RESULT_BACKEND,
             include=['celery_tasks.tasks'])

# Optional configuration, see the application user guide.
app.conf.update(
    result_expires=3600,
    task_serializer='json',
    accept_content=['json'],  # Ignore other content
    result_serializer='json',
    timezone='Asia/Shanghai',
    enable_utc=False,
)

if __name__ == '__main__':
    app.start()

3.2 tasks.py

# -*- coding: utf-8 -*-
# @Time    : 2019-04-30 12:00
# @Author  : kbsonlong
# @Email   : kbsonlong@gmail.com
# @Blog    : www.alongparty.cn
# @File    : tasks.py
# @Software: PyCharm

from __future__ import absolute_import, unicode_literals
from celery_tasks.celery import app


@app.task
def add(x, y):
    return x + y


@app.task
def mul(x, y):
    return x * y


@app.task
def xsum(numbers):
    return sum(numbers)

4 Django 项目配置 celery

vim testprj/testprj/init.py

from celery_tasks.celery import app as celery_app
__all__ = ['celery_app']

5 启动 celery work

# celery -A proj worker -l info 
-------------- celery@LAPTOP-Q26TVKSR v4.3.0 (rhubarb)
---- **** -----
--- * ***  * -- Windows-10-10.0.17134-SP0 2019-04-30 15:07:19
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         ops:0x17d108a5da0
- ** ---------- .> transport:   redis://localhost:6379/1
- ** ---------- .> results:     redis://localhost:6379/2
- *** --- * --- .> concurrency: 8 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . celery_tasks.tasks.add
  . celery_tasks.tasks.mul
  . celery_tasks.tasks.xsum

[2019-04-30 15:07:24,198: INFO/MainProcess] Connected to redis://localhost:6379/1
[2019-04-30 15:07:32,616: INFO/MainProcess] mingle: searching for neighbors
[2019-04-30 15:07:46,658: INFO/MainProcess] mingle: all alone
[2019-04-30 15:07:59,528: INFO/MainProcess] celery@LAPTOP-Q26TVKSR ready.
[2019-04-30 15:08:05,911: INFO/MainProcess] pidbox: Connected to redis://localhost:6379/1.
  • 从启动信息上可以看到有两部分内容,一部分为 celery 的配置信息,一部分为 tasks;
  • 从[tasks] 选项中可以看到刚才在 tasks.py 定义的 task 都加载进来了。

6 如何调用定义的任务

6.1 外部文件加载 tasks,如 views

from celery_tasks.tasks import add,mul,xsum

6.2 调用任务(task)

6.2.1 调用直接返回结果
add.delay(2,2)
6.2.2 异步调用
add.apply_async((2, 2))
6.2.3 异步调用,并在多久之后执行
res=add.apply_async((2, 2), queue='celery', countdown=10)
print(res.id,res.status)
  • 任务将被发送到一个名为的队列 celery,任务将在发送消息后 10 秒执行
  • 打印出任务 id 和状态
9f3a4f5b-c143-4fac-b26f-815df7ec3678 PENDING
  • 登录 redis,查看执行结果
{
status: "SUCCESS",
result: 4,
traceback: null,
children: [
],
task_id: "9f3a4f5b-c143-4fac-b26f-815df7ec3678",
date_done: "2019-04-30T07:52:32.543102"
}

运行 celery4.0 如果出现如下异常

  File "celery/backends/redis.py", line 22
    from . import async, base
                      ^
SyntaxError: invalid syntax
解决方案: pip install https://github.com/celery/celery/tarball/master
https://github.com/celery/celery/pull/4852/commits/d737dec3c943632f21f73a2235409c29e3fe63e3

参考资料

celery 官网
Django2 集成 celery4 执行异步任务、定时任务


蜷缩的蜗牛 , 版权所有丨如未注明 , 均为原创丨 转载请注明Django2 集成 celery4 执行异步任务、定时任务
喜欢 (0)
[]
分享 (0)

Django2集成celery4执行异步任务、定时任务


1 环境依赖

Python3.7

#依赖库
Django==2.1.7
celery==4.1.0
redis==3.2.1
#Windows 安装
eventlet (加入协程支持)

2 创建 django 项目

# django-admin startproject testprj
testprj/
├── manage.py
└── testprj
    ├── __init__.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py

1 directory, 5 files


3 创建 celery_tasks 模块

# cd testprj
# mkdir celery_tasks
tree celery_tasks/
celery_tasks/
├── celery.py
├── __init__.py
└── tasks.py

0 directories, 3 files

3.1 celery.py

# -*- coding: utf-8 -*-
# @Time    : 2019-04-30 11:55
# @Author  : kbsonlong
# @Email   : kbsonlong@gmail.com
# @Blog    : www.alongparty.cn
# @File    : celery.py
# @Software: PyCharm

from __future__ import absolute_import, unicode_literals
from celery import Celery

BROKER_URL = 'redis://localhost:6379/1'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/2'
app = Celery('testprj',
             broker=BROKER_URL,
             backend=CELERY_RESULT_BACKEND,
             include=['celery_tasks.tasks'])

# Optional configuration, see the application user guide.
app.conf.update(
    result_expires=3600,
    task_serializer='json',
    accept_content=['json'],  # Ignore other content
    result_serializer='json',
    timezone='Asia/Shanghai',
    enable_utc=False,
)

if __name__ == '__main__':
    app.start()

3.2 tasks.py

# -*- coding: utf-8 -*-
# @Time    : 2019-04-30 12:00
# @Author  : kbsonlong
# @Email   : kbsonlong@gmail.com
# @Blog    : www.alongparty.cn
# @File    : tasks.py
# @Software: PyCharm

from __future__ import absolute_import, unicode_literals
from celery_tasks.celery import app


@app.task
def add(x, y):
    return x + y


@app.task
def mul(x, y):
    return x * y


@app.task
def xsum(numbers):
    return sum(numbers)

4 Django 项目配置 celery

vim testprj/testprj/init.py

from celery_tasks.celery import app as celery_app
__all__ = ['celery_app']

5 启动 celery work

# celery -A proj worker -l info 
-------------- celery@LAPTOP-Q26TVKSR v4.3.0 (rhubarb)
---- **** -----
--- * ***  * -- Windows-10-10.0.17134-SP0 2019-04-30 15:07:19
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         ops:0x17d108a5da0
- ** ---------- .> transport:   redis://localhost:6379/1
- ** ---------- .> results:     redis://localhost:6379/2
- *** --- * --- .> concurrency: 8 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . celery_tasks.tasks.add
  . celery_tasks.tasks.mul
  . celery_tasks.tasks.xsum

[2019-04-30 15:07:24,198: INFO/MainProcess] Connected to redis://localhost:6379/1
[2019-04-30 15:07:32,616: INFO/MainProcess] mingle: searching for neighbors
[2019-04-30 15:07:46,658: INFO/MainProcess] mingle: all alone
[2019-04-30 15:07:59,528: INFO/MainProcess] celery@LAPTOP-Q26TVKSR ready.
[2019-04-30 15:08:05,911: INFO/MainProcess] pidbox: Connected to redis://localhost:6379/1.
  • 从启动信息上可以看到有两部分内容,一部分为 celery 的配置信息,一部分为 tasks;
  • 从[tasks] 选项中可以看到刚才在 tasks.py 定义的 task 都加载进来了。

6 如何调用定义的任务

6.1 外部文件加载 tasks,如 views

from celery_tasks.tasks import add,mul,xsum

6.2 调用任务(task)

6.2.1 调用直接返回结果
add.delay(2,2)
6.2.2 异步调用
add.apply_async((2, 2))
6.2.3 异步调用,并在多久之后执行
res=add.apply_async((2, 2), queue='celery', countdown=10)
print(res.id,res.status)
  • 任务将被发送到一个名为的队列 celery,任务将在发送消息后 10 秒执行
  • 打印出任务 id 和状态
9f3a4f5b-c143-4fac-b26f-815df7ec3678 PENDING
  • 登录 redis,查看执行结果
{
status: "SUCCESS",
result: 4,
traceback: null,
children: [
],
task_id: "9f3a4f5b-c143-4fac-b26f-815df7ec3678",
date_done: "2019-04-30T07:52:32.543102"
}

运行 celery4.0 如果出现如下异常

  File "celery/backends/redis.py", line 22
    from . import async, base
                      ^
SyntaxError: invalid syntax
解决方案: pip install https://github.com/celery/celery/tarball/master
https://github.com/celery/celery/pull/4852/commits/d737dec3c943632f21f73a2235409c29e3fe63e3

参考资料

celery 官网


蜷缩的蜗牛 , 版权所有丨如未注明 , 均为原创丨 转载请注明Django2 集成 celery4 执行异步任务、定时任务
喜欢 (0)
[]
分享 (0)
蜷缩的蜗牛
关于作者:
IT运维工程师,熟练使用复制、粘贴技能;平时爱好打羽毛球,虽然很菜,也喜欢出去旅游!个人公众号:蜷缩的蜗牛,欢迎关注订阅~~网站小程序:alongparty,支持搜索博客内容,订阅专题,评论、点赞等等

您必须 登录 才能发表评论!