Skip to content
This repository has been archived by the owner on Nov 23, 2017. It is now read-only.

ThirdParty

Jump to bottom
Victor Stinner edited this page Nov 17, 2017 · 57 revisions

A list of libraries for AsyncIO is available on PyPI with Framework::AsyncIO classifier.

Alternative event loop implementations

  • uvloop: Ultra fast implementation of asyncio event loop on top of libuv. Read more about uvloop here.
  • async-tokio: Asyncio event loop implementation, written in Rust language

Libraries

  • aiofiles: File support for asyncio
  • aiocache: Caching for asyncio with multiple backends (framework agnostic)
  • aiorwlock: Read write lock for asyncio
  • aioutils: Python3 Asyncio Utils, Group (like gevent.pool.Group), Pool (like gevent.pool.Pool), Bag and OrderedBag
  • tasklocals: Task-local variables
  • janus: Thread-safe asyncio-aware queue
  • asyncio_extras: asynchronous generator/context manager support, thread pool integration, async file operations and more
  • paco: Small utility library for coroutine-driven asynchronous programming
  • aioconsole: Asynchronous console and interfaces for asyncio
  • aiostream: Generator-based operators for asynchronous iteration

Unit testing

  • asynctest: Enhance the standard unittest package for testing asyncio libraries
  • pytest-asyncio: Pytest support for asyncio

Protocols implementations

Clients

  • AMQP:
    • aioamqp: AMQP implementation using asyncio (Used by RabbitMQ)
    • QBroker: easy-to-use library based on aioamqp, for pub/sub and RPC
  • AMI: panoramisk, a library to play with Asterisk's protocols: AMI and FastAGI
  • Cassandra: aiocassandra, Simple threaded cassandra wrapper for asyncio
  • CouchDB: aiocouchdb, CouchDB client
  • DNS: aiodns: Async DNS resolver
  • ElasticSearch:
  • Etcd: aioetcd: Coroutine-based etcd client
  • Google Hangouts: hangups: Client for Google Hangouts
  • HTTP: aiohttp.requests: http client with Requests-like API.
  • IRC:
    • irc3, plugable irc client library based on python's asyncio
    • bottom, asyncio-based rfc2812-compliant IRC Client
  • Kafka: aiokafka: Full featured Kafka client for Kafka 9.0.0+
  • memcached: aiomemcache, minimal memcached client
  • MongoDB:
    • Motor, official async MongoDB driver
    • asyncio-mongo, MongoDB driver (ported from Twisted)
  • MPD: aiompd, Music Player Daemon client for AsyncIO
  • MySQL: aiomysql, MySQL driver
  • ODBC: aioodbc, ODBC client on top of pyodbc, works only with python 3.5+
  • PostgreSQL:
    • aiopg, PostgreSQL client library built on top of psycopg2
    • aiopg8000, A Pure-Python PostgreSQL driver for AsyncIO
    • asyncpg, A fast PostgreSQL Database Client Library for Python/asyncio
  • Prometheus: prometheus_async adds support for asynchronous frameworks to the official Python client for the Prometheus metrics and monitoring system.
  • PySerial: asyncserial: asyncio support for pySerial
  • Redis:
  • Riak: aioriak: Python asyncio client for Riak
  • Slack aioslacker: slacker wrapper for asyncio
  • SIP: aiosip: SIP support for AsyncIO
  • Socks: aiosocks: SOCKS proxy client for asyncio and aiohttp
  • SSH: AsyncSSH: SSH client and server implementation
  • WebSockets: aiohttp.ws_connect: WebSockets client
  • XMPP (Jabber):
    • slixmpp, SleekXMPP (XMPP Library) fork using asyncio, for poezio,
    • aioxmpp, from scratch XMPP library for asyncio (alpha)
  • Zipkin: aiozipkin: Distributed tracing instrumentation for asyncio with zipkin

Servers

  • FastAGI: panoramisk, a library to play with Asterisk's protocols: AMI and FastAGI
  • IRC: irc3d, irc server library based on irc3
  • HTTP: aiohttp: http client and server infrastructure for asyncio
  • SSH: AsyncSSH: SSH client and server implementation
  • SMTP/LMTP: aiosmtpd: Server implementations and testing frameworks for Simple Mail Transport Protocol and Local Mail Transport Protocol.

Web Frameworks

For classical HTTP/1.1 protocol

Looking for WSGI? Read this thread: WSGI implementation compatible with asyncio?.

  • aiohttp.web: a Flask-like API to build quickly HTTP applications, made by the creators of aiohttp
  • aiopyramid: Tools for running pyramid using asyncio
  • aiotraversal: Web framework for big complicated applications based on aiohttp
  • aiowsgi: minimalist wsgi server using asyncio
  • Growler: A web framework built on top of aiohttp
  • interest: event-driven web framework on top of aiohttp/asyncio
  • muffin: A web framework based on Asyncio stack (early alpha)
  • nacho: web framework
  • Pulsar: Event driven concurrent framework for python. With pulsar you can write asynchronous servers performing one or several activities in different threads and/or processes.
  • rainfall: another web framework
  • sanic: A Flask-like Python 3.5+ web server that's written to go fast.
  • Vase: web framework

For WebSockets

ORMs

  • aiopg.sa: support for SQLAlchemy functional SQL layer, based on aiopg
  • aiomysql.sa: support for SQLAlchemy functional SQL layer, based on aiomysql
  • peewee: peewee-async, library providing asynchronous interface powered by asyncio for peewee ORM.
  • GINO: lightweight ORM based on asyncpg and SQLAlchemy core

Integration with other applications

  • API-Hour: Transform easily your AsyncIO server source code (HTTP, SSH, ...) to be multiprocess: it will help to improve the efficency on multi-CPU servers.
  • Gunicorn: Gunicorn has gaiohttp worker built on top of aiohttp library

Applications built with AsyncIO

  • ktcal2: SSH brute forcer tool and library, using AsyncIO of Python 3.4

Run asyncio on top of

  • eventlet: aioeventlet, asyncio API implemented on top of eventlet

Adapters for other event loops

Some people have already written adapters for integrating asyncio with other async I/O frameworks.

Misc

  • aioauth-client: OAuth1/2 support (authorization, resource's loading)
  • aiocron: Crontabs for asyncio
  • aiomas: A library for multi-agent systems and RPC based on asyncio
  • aiotest: test suite to validate an implementation of the asyncio API
  • aioprocessing: A Python 3.3+ library that integrates the multiprocessing module with asyncio
  • blender-asyncio: Asyncio Bridge for Blender Python API
  • ipython-yf: An ipython extension to make it asyncio compatible
  • aiogearman: asyncio gearman support. Gearman provides a generic application framework to farm out work to other machines or processes that are better suited to do the work.
  • aiogear: Yet another gearman protocol implemented in asyncio.
  • Serial port using the serial module, see using serial port in python3 asyncio at Stackoverflow, serial.Serial can be registered with loop.add_reader().
  • async_lru: port of functools.lru_cache to asyncio
  • GoogleScraper: A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, Baidu and others) by using proxies (socks4/5, http proxy) and with many different IP's.
  • aiotasks: A Celery like task manager that distributes Asyncio coroutines.

Filesystem

asyncio does not support asynchronous operations on the filesystem. Even if files are opened with O_NONBLOCK, read and write will block.

Read asynchronous disk I/O (October 2012 by arvid).

The Linux kernel provides asynchronous operations on the filesystem (aio), but it requires a library and it doesn't scale with many concurrent operations. See aio.

The GNU C library (glibc) implements the POSIX aio interface, but it is implemented with threads. See aio(7) manual page.

Recent discussion on the Linux Kernel: Non-blocking buffered file read operations (September 2014).

For now, the workaround is to use aiofiles that uses threads to handle files.

Clone this wiki locally