wsproto: A pure Python WebSocket protocol stack

wsproto is a WebSocket protocol stack written to be as flexible as possible. To that end it is written in pure Python and performs no I/O of its own. Instead it relies on the user to provide a bridge between it and whichever I/O mechanism is in use, allowing it to be used in single-threaded, multi-threaded or event-driven code.

The goal for wsproto is 100% compliance with RFC 6455. Additionally a mechanism is provided to add extensions allowing the implementation of extra functionally such as per-message compression as specified in RFC 7692.

For usage examples, see Getting Started or see the examples provided.

Contents:

Installation

wsproto is a pure Python project. To install it you can use pip like so:

$ pip install wsproto

Alternatively you can get either a release tarball or a development branch from our GitHub repository and run:

$ python setup.py install

Getting Started

This document explains how to get started using wsproto to connect to WebSocket servers as well as how to write your own.

We assume some level of familiarity with writing Python and networking code. If you’re not familiar with these we highly recommend you read up on these first. It may also be helpful to study Sans-I/O, which describes the ideas behind writing a network protocol library that doesn’t do any network I/O.

Connections

The main class you’ll be working with is the WSConnection object. This object represents a connection to a WebSocket client or server and contains all the state needed to communicate with the entity at the other end. Whether you’re connecting to a server or receiving a connection from a client, this is the object you’ll use.

wsproto provides two layers of abstractions. You need to write code that interfaces with both of these layers. The following diagram illustrates how your code is like a sandwich around wsproto.

Application

<APPLICATION GLUE>

wsproto

<NETWORK GLUE>

Network Layer

wsproto does not do perform any network I/O, so <NETWORK GLUE> represents the code you need to write to glue wsproto to the actual network layer, i.e. code that can send and receive data over the network. The WSConnection class provides two methods for this purpose. When data has been received on a network socket, you feed this data into wsproto by calling receive_bytes. When wsproto has data that needs to be sent over the network, you retrieve that data by calling bytes_to_send, and your code is responsible for actually sending that data over the network.

Note

If the connection drops, a standard Python socket.recv() will return zero. You should call receive_bytes(None) to update the internal wsproto state to indicate that the connection has been closed.

Internally, wsproto process the raw network data you feed into it and turns it into higher level representations of WebSocket events. In <APPLICATION GLUE>, you need to write code to process these events. The WSConnection class contains a generator method events that yields WebSocket events. To send a message, you call the send_data method.

Connecting to a WebSocket server

Begin by instantiating a connection object. The host and resource arguments are required to instantiate a client. If the WebSocket server is located at http://myhost.com/foo, then you would instantiate the connection as follows:

ws = WSConnection(ConnectionType.CLIENT, host="myhost.com", resource='foo')

Now you need to provide the network glue. For the sake of example, we will use standard Python sockets here, but wsproto can be integrated with any network layer:

stream = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
stream.connect(("myhost", 8000))

To read from the network:

data = stream.recv(4096)
ws.receive_bytes(data)

You also need to check if wsproto has data to send to the network:

data = ws.bytes_to_send()
stream.send(data)

Note that bytes_to_send() will return zero bytes if the protocol has no pending data. You can either poll this method or call it only when you expect to have pending data.

A standard Python socket will block on the call to stream.recv(), so you will probably need to use a non-blocking socket or some form of concurrency like threading, greenlets, asyncio, etc.

You also need to provide the application glue. To send a WebSocket message:

ws.send_data("Hello world!")

And to receive WebSocket events:

for event in ws.events():
    if isinstance(event, ConnectionEstablished):
        print('Connection established')
    elif isinstance(event, ConnectionClosed):
        print('Connection closed: code={} reason={}'.format(
            event.code, event.reason))
    elif isinstance(event, TextReceived):
        print('Received message: {}'.format(event.data))
    else:
        print('Unknown event: {!r}'.format(event))

The method events() returns a generator which will yield events for all of the data currently in the wsproto internal buffer and then exit. Therefore, you should iterate over this generator after receiving new network data.

For a more complete example, see synchronous_client.py.

WebSocket Servers

A WebSocket server is similar to a client except that it uses a different constant:

ws = WSConnection(ConnectionType.SERVER)

A server also needs to explicitly call the accept method after it receives a ConnectionRequested event:

for event in ws.events():
    if isinstance(event, ConnectionRequested):
        print('Accepting connection request')
        ws.accept(event)
    elif isinstance(event, ConnectionClosed):
        print('Connection closed: code={} reason={}'.format(
            event.code, event.reason))
    elif isinstance(event, TextReceived):
        print('Received message: {}'.format(event.data))
    else:
        print('Unknown event: {!r}'.format(event))

For a more complete example, see synchronous_server.py.

Closing

WebSockets are closed with a handshake that requires each endpoint to send one frame and receive one frame. The close() method places a close frame in the send buffer. When a close frame is received, it yields a ConnectionClosed event, and it also places a reply frame in the send buffer. When that reply has been received by the initiator, it will also receive a ConnectionClosed event.

Regardless of which endpoint initiates the closing handshake, the server is responsible for tearing down the underlying connection. When the server receives a ConnectionClosed event, it should send pending wsproto data (if any) and then it can start tearing down the underlying connection.

Ping Pong

The WSConnection class supports sending WebSocket ping and pong frames via the methods ping and pong.

Note

When a ping is received, wsproto automatically places a pong frame in its outgoing buffer. You should only call pong() if you want to send an unsolicited pong frame.

Back-pressure

Back-pressure is an important concept to understand when implementing a client/server protocol. This section briefly explains the issue and then explains how to handle back-pressure when using wsproto.

Imagine that you have a WebSocket server that reads messages from the client, does some processing, and then sends a response. What happens if the client sends messages faster than the the server can process them? If the incoming messages are buffered in memory, then the server will slowly use more and more memory, until the OS eventually kills it. This scenario is directly applicable to wsproto, because every time you call receive_bytes(), it appends that data to an internal buffer.

The slow endpoint needs a way to signal the fast endpoint to stop sending messages until the slow endpoint can catch up. This signaling is called “back-pressure”. As a Sans-IO library, wsproto is not responsible for network concerns like back-pressure, so that responsibility belongs to your network glue code.

Fortunately, TCP has the ability to signal backpressure, and the operating system will do that for you automatically—if you follow a few rules! The OS buffers all incoming and outgoing network data. Standard Python socket methods like send() and recv() copy data to and from those OS buffers. For example, if the peer is sending data too quickly, then the OS receive buffere will start to get full, and the OS will signal the peer to stop transmitting. When recv() is called, the OS will copy data from its internal buffer into your process, free up space in its own buffer, and then signal to the peer to start transmitting again.

Therefore, you need to follow these two rules to implement back-pressure over TCP:

1. Do not receive from the socket faster than your code can process the messages. Your processing code may need to signal the receiving code when its ready to receive more data.
2. Do not store out-going messages in an unbounded collection. Ideally, out-going messages should be sent to the OS as soon as possible. If you need to buffer messages in memory, the buffer should be bounded so that it can not grow indefinitely.

wsproto API

This document details the API of wsproto.

Semantic Versioning

wsproto follows semantic versioning for its public API. Please note that the guarantees of semantic versioning apply only to the API that is documented here. Simply because a method or data field is not prefaced by an underscore does not make it part of wsproto’s public API. Anything not documented here is subject to change at any time.

Connection

class wsproto.connection.WSConnection(conn_type, host=None, resource=None, extensions=None, subprotocols=None)

A low-level WebSocket connection object.

This wraps two other protocol objects, an HTTP/1.1 protocol object used to do the initial HTTP upgrade handshake and a WebSocket frame protocol object used to exchange messages and other control frames.

Parameters:

• conn_type (ConnectionType) – Whether this object is on the client- or server-side of a connection. To initialise as a client pass CLIENT otherwise pass SERVER.
• host (str) – The hostname to pass to the server when acting as a client.
• resource (str) – The resource (aka path) to pass to the server when acting as a client.
• extensions – A list of extensions to use on this connection. Defaults to to an empty list. Extensions should be instances of a subclass of Extension.
• subprotocols – A list of subprotocols to request when acting as a client, ordered by preference. This has no impact on the connection itself. Defaults to an empty list.

bytes_to_send(amount=None)

Returns some data for sending out of the internal data buffer.

This method is analogous to read on a file-like object, but it doesn’t block. Instead, it returns as much data as the user asks for, or less if that much data is not available. It does not perform any I/O, and so uses a different name.

Parameters:amount (int) – (optional) The maximum amount of data to return. If not set, or set to None, will return as much data as possible. Returns:A bytestring containing the data to send on the wire. Return type:bytes

close(code=<CloseReason.NORMAL_CLOSURE: 1000>, reason=None)

Initiate the close handshake by sending a CLOSE control message.

A clean teardown requires a CLOSE control messages from the other endpoint before the underlying TCP connection can be closed, see ConnectionClosed.

events()

Return a generator that provides any events that have been generated by protocol activity.

Returns:generator of Event subclasses

ping(payload=None)

Send a PING message to the peer.

Parameters:payload – an optional payload to send with the message

pong(payload=None)

Send a PONG message to the peer.

This method can be used to send an unsolicted PONG to the peer. It is not needed otherwise since every received PING causes a corresponding PONG to be sent automatically.

Parameters:payload – an optional payload to send with the message

receive_bytes(data)

Pass some received data to the connection for handling.

A list of events that the remote peer triggered by sending this data can be retrieved with events().

Parameters:data (bytes) – The data received from the remote peer on the network.

send_data(payload, final=True)

Send a message or part of a message to the remote peer.

If final is False it indicates that this is part of a longer message. If final is True it indicates that this is either a self-contained message or the last part of a longer message.

If payload is of type bytes then the message is flagged as being binary. If it is of type str the message is encoded as UTF-8 and sent as text.

Parameters:

• payload (bytes or str) – The message body to send.
• final (bool) – Whether there are more parts to this message to be sent.

Events

class wsproto.events.Event

Base class for wsproto events.

class wsproto.events.ConnectionRequested(proposed_subprotocols, h11request)

class wsproto.events.ConnectionEstablished(subprotocol=None, extensions=None)

class wsproto.events.ConnectionClosed(code, reason=None)

The ConnectionClosed event is fired after the connection is considered closed.

wsproto automatically emits a CLOSE frame when it receives one, to complete the close-handshake.

code = None

The close status code, see CloseReason.

class wsproto.events.ConnectionFailed(code, reason=None)

class wsproto.events.DataReceived(data, frame_finished, message_finished)

class wsproto.events.TextReceived(data, frame_finished, message_finished)

class wsproto.events.BytesReceived(data, frame_finished, message_finished)

class wsproto.events.PingReceived(payload)

class wsproto.events.PongReceived(payload)

Frame Protocol

class wsproto.frame_protocol.Opcode

RFC 6455, Section 5.2 - Base Framing Protocol

class wsproto.frame_protocol.CloseReason

RFC 6455, Section 7.4.1 - Defined Status Codes

Extensions

class wsproto.extensions.Extension

wsproto.extensions.SUPPORTED_EXTENSIONS = {'permessage-deflate': <class 'wsproto.extensions.PerMessageDeflate'>}

SUPPORTED_EXTENSIONS maps all supported extension names to their class. This can be used to iterate all supported extensions of wsproto, instantiate new extensions based on their name, or check if a given extension is supported or not.