Negotiating HTTP/2

RFC 7540 specifies three methods of negotiating HTTP/2 connections. This document outlines how to use h2 with each one.

HTTPS URLs (ALPN)

Starting HTTP/2 for HTTPS URLs is outlined in RFC 7540 Section 3.3. In this case, the client and server use a TLS extension to negotiate HTTP/2: ALPN. How to use ALPN is currently not covered in this document: please consult the documentation for either the ssl module in the standard library, or the PyOpenSSL third-party modules, for more on this topic.

This method is the simplest to use once the TLS connection is established. To use it with h2, after you’ve established the connection and confirmed that HTTP/2 has been negotiated with ALPN, create a H2Connection object and call H2Connection.initiate_connection. This will ensure that the appropriate preamble data is placed in the data buffer. You should then immediately send the data returned by H2Connection.data_to_send on your TLS connection.

At this point, you’re free to use all the HTTP/2 functionality provided by h2.

Note

Although h2 is not concerned with negotiating protocol versions, it is important to note that support for ALPN is not available in the standard library of Python versions < 2.7.9. As a consequence, clients may encounter various errors due to protocol versions mismatch.

Server Setup Example

This example uses the APIs as defined in Python 3.5. If you are using an older version of Python you may not have access to the APIs used here. As noted above, please consult the documentation for the ssl module to confirm.

  1# -*- coding: utf-8 -*-
  2"""
  3Server HTTPS Setup
  4~~~~~~~~~~~~~~~~~~
  5
  6This example code fragment demonstrates how to set up a HTTP/2 server that
  7negotiates HTTP/2 using NPN and ALPN. For the sake of maximum explanatory value
  8this code uses the synchronous, low-level sockets API: however, if you're not
  9using sockets directly (e.g. because you're using asyncio), you should focus on
 10the set up required for the SSLContext object. For other concurrency libraries
 11you may need to use other setup (e.g. for Twisted you'll need to use
 12IProtocolNegotiationFactory).
 13
 14This code requires Python 3.5 or later.
 15"""
 16import h2.config
 17import h2.connection
 18import socket
 19import ssl
 20
 21
 22def establish_tcp_connection():
 23    """
 24    This function establishes a server-side TCP connection. How it works isn't
 25    very important to this example.
 26    """
 27    bind_socket = socket.socket()
 28    bind_socket.bind(('', 443))
 29    bind_socket.listen(5)
 30    return bind_socket.accept()[0]
 31
 32
 33def get_http2_ssl_context():
 34    """
 35    This function creates an SSLContext object that is suitably configured for
 36    HTTP/2. If you're working with Python TLS directly, you'll want to do the
 37    exact same setup as this function does.
 38    """
 39    # Get the basic context from the standard library.
 40    ctx = ssl.create_default_context(purpose=ssl.Purpose.CLIENT_AUTH)
 41
 42    # RFC 7540 Section 9.2: Implementations of HTTP/2 MUST use TLS version 1.2
 43    # or higher. Disable TLS 1.1 and lower.
 44    ctx.options |= (
 45        ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3 | ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1
 46    )
 47
 48    # RFC 7540 Section 9.2.1: A deployment of HTTP/2 over TLS 1.2 MUST disable
 49    # compression.
 50    ctx.options |= ssl.OP_NO_COMPRESSION
 51
 52    # RFC 7540 Section 9.2.2: "deployments of HTTP/2 that use TLS 1.2 MUST
 53    # support TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256". In practice, the
 54    # blocklist defined in this section allows only the AES GCM and ChaCha20
 55    # cipher suites with ephemeral key negotiation.
 56    ctx.set_ciphers("ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM:DHE+CHACHA20")
 57
 58    # We want to negotiate using NPN and ALPN. ALPN is mandatory, but NPN may
 59    # be absent, so allow that. This setup allows for negotiation of HTTP/1.1.
 60    ctx.set_alpn_protocols(["h2", "http/1.1"])
 61
 62    try:
 63        ctx.set_npn_protocols(["h2", "http/1.1"])
 64    except NotImplementedError:
 65        pass
 66
 67    return ctx
 68
 69
 70def negotiate_tls(tcp_conn, context):
 71    """
 72    Given an established TCP connection and a HTTP/2-appropriate TLS context,
 73    this function:
 74
 75    1. wraps TLS around the TCP connection.
 76    2. confirms that HTTP/2 was negotiated and, if it was not, throws an error.
 77    """
 78    tls_conn = context.wrap_socket(tcp_conn, server_side=True)
 79
 80    # Always prefer the result from ALPN to that from NPN.
 81    # You can only check what protocol was negotiated once the handshake is
 82    # complete.
 83    negotiated_protocol = tls_conn.selected_alpn_protocol()
 84    if negotiated_protocol is None:
 85        negotiated_protocol = tls_conn.selected_npn_protocol()
 86
 87    if negotiated_protocol != "h2":
 88        raise RuntimeError("Didn't negotiate HTTP/2!")
 89
 90    return tls_conn
 91
 92
 93def main():
 94    # Step 1: Set up your TLS context.
 95    context = get_http2_ssl_context()
 96
 97    # Step 2: Receive a TCP connection.
 98    connection = establish_tcp_connection()
 99
100    # Step 3: Wrap the connection in TLS and validate that we negotiated HTTP/2
101    tls_connection = negotiate_tls(connection, context)
102
103    # Step 4: Create a server-side H2 connection.
104    config = h2.config.H2Configuration(client_side=False)
105    http2_connection = h2.connection.H2Connection(config=config)
106
107    # Step 5: Initiate the connection
108    http2_connection.initiate_connection()
109    tls_connection.sendall(http2_connection.data_to_send())
110
111    # The TCP, TLS, and HTTP/2 handshakes are now complete. You can enter your
112    # main loop now.

Client Setup Example

The client example is very similar to the server example above. The SSLContext object requires some minor changes, as does the H2Connection, but the bulk of the code is the same.

  1# -*- coding: utf-8 -*-
  2"""
  3Client HTTPS Setup
  4~~~~~~~~~~~~~~~~~~
  5
  6This example code fragment demonstrates how to set up a HTTP/2 client that
  7negotiates HTTP/2 using NPN and ALPN. For the sake of maximum explanatory value
  8this code uses the synchronous, low-level sockets API: however, if you're not
  9using sockets directly (e.g. because you're using asyncio), you should focus on
 10the set up required for the SSLContext object. For other concurrency libraries
 11you may need to use other setup (e.g. for Twisted you'll need to use
 12IProtocolNegotiationFactory).
 13
 14This code requires Python 3.5 or later.
 15"""
 16import h2.connection
 17import socket
 18import ssl
 19
 20
 21def establish_tcp_connection():
 22    """
 23    This function establishes a client-side TCP connection. How it works isn't
 24    very important to this example. For the purpose of this example we connect
 25    to localhost.
 26    """
 27    return socket.create_connection(('localhost', 443))
 28
 29
 30def get_http2_ssl_context():
 31    """
 32    This function creates an SSLContext object that is suitably configured for
 33    HTTP/2. If you're working with Python TLS directly, you'll want to do the
 34    exact same setup as this function does.
 35    """
 36    # Get the basic context from the standard library.
 37    ctx = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH)
 38
 39    # RFC 7540 Section 9.2: Implementations of HTTP/2 MUST use TLS version 1.2
 40    # or higher. Disable TLS 1.1 and lower.
 41    ctx.options |= (
 42        ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3 | ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1
 43    )
 44
 45    # RFC 7540 Section 9.2.1: A deployment of HTTP/2 over TLS 1.2 MUST disable
 46    # compression.
 47    ctx.options |= ssl.OP_NO_COMPRESSION
 48
 49    # RFC 7540 Section 9.2.2: "deployments of HTTP/2 that use TLS 1.2 MUST
 50    # support TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256". In practice, the
 51    # blocklist defined in this section allows only the AES GCM and ChaCha20
 52    # cipher suites with ephemeral key negotiation.
 53    ctx.set_ciphers("ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM:DHE+CHACHA20")
 54
 55    # We want to negotiate using NPN and ALPN. ALPN is mandatory, but NPN may
 56    # be absent, so allow that. This setup allows for negotiation of HTTP/1.1.
 57    ctx.set_alpn_protocols(["h2", "http/1.1"])
 58
 59    try:
 60        ctx.set_npn_protocols(["h2", "http/1.1"])
 61    except NotImplementedError:
 62        pass
 63
 64    return ctx
 65
 66
 67def negotiate_tls(tcp_conn, context):
 68    """
 69    Given an established TCP connection and a HTTP/2-appropriate TLS context,
 70    this function:
 71
 72    1. wraps TLS around the TCP connection.
 73    2. confirms that HTTP/2 was negotiated and, if it was not, throws an error.
 74    """
 75    # Note that SNI is mandatory for HTTP/2, so you *must* pass the
 76    # server_hostname argument.
 77    tls_conn = context.wrap_socket(tcp_conn, server_hostname='localhost')
 78
 79    # Always prefer the result from ALPN to that from NPN.
 80    # You can only check what protocol was negotiated once the handshake is
 81    # complete.
 82    negotiated_protocol = tls_conn.selected_alpn_protocol()
 83    if negotiated_protocol is None:
 84        negotiated_protocol = tls_conn.selected_npn_protocol()
 85
 86    if negotiated_protocol != "h2":
 87        raise RuntimeError("Didn't negotiate HTTP/2!")
 88
 89    return tls_conn
 90
 91
 92def main():
 93    # Step 1: Set up your TLS context.
 94    context = get_http2_ssl_context()
 95
 96    # Step 2: Create a TCP connection.
 97    connection = establish_tcp_connection()
 98
 99    # Step 3: Wrap the connection in TLS and validate that we negotiated HTTP/2
100    tls_connection = negotiate_tls(connection, context)
101
102    # Step 4: Create a client-side H2 connection.
103    http2_connection = h2.connection.H2Connection()
104
105    # Step 5: Initiate the connection
106    http2_connection.initiate_connection()
107    tls_connection.sendall(http2_connection.data_to_send())
108
109    # The TCP, TLS, and HTTP/2 handshakes are now complete. You can enter your
110    # main loop now.

HTTP URLs (Upgrade)

Starting HTTP/2 for HTTP URLs is outlined in RFC 7540 Section 3.2. In this case, the client and server use the HTTP Upgrade mechanism originally described in RFC 7230 Section 6.7. The client sends its initial HTTP/1.1 request with two extra headers. The first is Upgrade: h2c, which requests upgrade to cleartext HTTP/2. The second is a HTTP2-Settings header, which contains a specially formatted string that encodes a HTTP/2 Settings frame.

To do this with h2 you have two slightly different flows: one for clients, one for servers.

Clients

For a client, when sending the first request you should manually add your Upgrade header. You should then create a H2Connection object and call H2Connection.initiate_upgrade_connection with no arguments. This method will return a bytestring to use as the value of your HTTP2-Settings header.

If the server returns a 101 status code, it has accepted the upgrade, and you should immediately send the data returned by H2Connection.data_to_send. Now you should consume the entire 101 header block. All data after the 101 header block is HTTP/2 data that should be fed directly to H2Connection.receive_data and handled as normal with h2.

If the server does not return a 101 status code then it is not upgrading. Continue with HTTP/1.1 as normal: you may throw away your H2Connection object, as it is of no further use.

The server will respond to your original request in HTTP/2. Please pay attention to the events received from h2, as they will define the server’s response.

Client Example

The code below demonstrates how to handle a plaintext upgrade from the perspective of the client. For the purposes of keeping the example code as simple and generic as possible it uses the synchronous socket API that comes with the Python standard library: if you want to use asynchronous I/O, you will need to translate this code to the appropriate idiom.

  1# -*- coding: utf-8 -*-
  2"""
  3Client Plaintext Upgrade
  4~~~~~~~~~~~~~~~~~~~~~~~~
  5
  6This example code fragment demonstrates how to set up a HTTP/2 client that uses
  7the plaintext HTTP Upgrade mechanism to negotiate HTTP/2 connectivity. For
  8maximum explanatory value it uses the synchronous socket API that comes with
  9the Python standard library. In product code you will want to use an actual
 10HTTP/1.1 client if possible.
 11
 12This code requires Python 3.5 or later.
 13"""
 14import h2.connection
 15import socket
 16
 17
 18def establish_tcp_connection():
 19    """
 20    This function establishes a client-side TCP connection. How it works isn't
 21    very important to this example. For the purpose of this example we connect
 22    to localhost.
 23    """
 24    return socket.create_connection(('localhost', 80))
 25
 26
 27def send_initial_request(connection, settings):
 28    """
 29    For the sake of this upgrade demonstration, we're going to issue a GET
 30    request against the root of the site. In principle the best request to
 31    issue for an upgrade is actually ``OPTIONS *``, but this is remarkably
 32    poorly supported and can break in weird ways.
 33    """
 34    # Craft our initial request per RFC 7540 Section 3.2. This requires two
 35    # special header fields: the Upgrade headre, and the HTTP2-Settings header.
 36    # The value of the HTTP2-Settings header field comes from h2.
 37    request = (
 38        b"GET / HTTP/1.1\r\n" +
 39        b"Host: localhost\r\n" +
 40        b"Upgrade: h2c\r\n" +
 41        b"HTTP2-Settings: " + settings + b"\r\n" +
 42        b"\r\n"
 43    )
 44    connection.sendall(request)
 45
 46
 47def get_upgrade_response(connection):
 48    """
 49    This function reads from the socket until the HTTP/1.1 end-of-headers
 50    sequence (CRLFCRLF) is received. It then checks what the status code of the
 51    response is.
 52
 53    This is not a substitute for proper HTTP/1.1 parsing, but it's good enough
 54    for example purposes.
 55    """
 56    data = b''
 57    while b'\r\n\r\n' not in data:
 58        data += connection.recv(8192)
 59
 60    headers, rest = data.split(b'\r\n\r\n', 1)
 61
 62    # An upgrade response begins HTTP/1.1 101 Switching Protocols. Look for the
 63    # code. In production code you should also check that the upgrade is to
 64    # h2c, but here we know we only offered one upgrade so there's only one
 65    # possible upgrade in use.
 66    split_headers = headers.split()
 67    if split_headers[1] != b'101':
 68        raise RuntimeError("Not upgrading!")
 69
 70    # We don't care about the HTTP/1.1 data anymore, but we do care about
 71    # any other data we read from the socket: this is going to be HTTP/2 data
 72    # that must be passed to the H2Connection.
 73    return rest
 74
 75
 76def main():
 77    """
 78    The client upgrade flow.
 79    """
 80    # Step 1: Establish the TCP connection.
 81    connection = establish_tcp_connection()
 82
 83    # Step 2: Create H2 Connection object, put it in upgrade mode, and get the
 84    # value of the HTTP2-Settings header we want to use.
 85    h2_connection = h2.connection.H2Connection()
 86    settings_header_value = h2_connection.initiate_upgrade_connection()
 87
 88    # Step 3: Send the initial HTTP/1.1 request with the upgrade fields.
 89    send_initial_request(connection, settings_header_value)
 90
 91    # Step 4: Read the HTTP/1.1 response, look for 101 response.
 92    extra_data = get_upgrade_response(connection)
 93
 94    # Step 5: Immediately send the pending HTTP/2 data.
 95    connection.sendall(h2_connection.data_to_send())
 96
 97    # Step 6: Feed the body data to the connection.
 98    events = connection.receive_data(extra_data)
 99
100    # Now you can enter your main loop, beginning by processing the first set
101    # of events above. These events may include ResponseReceived, which will
102    # contain the response to the request we made in Step 3.
103    main_loop(events)

Servers

If the first request you receive on a connection from the client contains an Upgrade header with the h2c token in it, and you’re willing to upgrade, you should create a H2Connection object and call H2Connection.initiate_upgrade_connection with the value of the HTTP2-Settings header (as a bytestring) as the only argument.

Then, you should send back a 101 response that contains h2c in the Upgrade header. That response will inform the client that you’re switching to HTTP/2. Then, you should immediately send the data that is returned to you by H2Connection.data_to_send on the connection: this is a necessary part of the HTTP/2 upgrade process.

At this point, you may now respond to the original HTTP/1.1 request in HTTP/2 by calling the appropriate methods on the H2Connection object. No further HTTP/1.1 may be sent on this connection: from this point onward, all data sent by you and the client will be HTTP/2 data.

Server Example

The code below demonstrates how to handle a plaintext upgrade from the perspective of the server. For the purposes of keeping the example code as simple and generic as possible it uses the synchronous socket API that comes with the Python standard library: if you want to use asynchronous I/O, you will need to translate this code to the appropriate idiom.

  1# -*- coding: utf-8 -*-
  2"""
  3Server Plaintext Upgrade
  4~~~~~~~~~~~~~~~~~~~~~~~~
  5
  6This example code fragment demonstrates how to set up a HTTP/2 server that uses
  7the plaintext HTTP Upgrade mechanism to negotiate HTTP/2 connectivity. For
  8maximum explanatory value it uses the synchronous socket API that comes with
  9the Python standard library. In product code you will want to use an actual
 10HTTP/1.1 server library if possible.
 11
 12This code requires Python 3.5 or later.
 13"""
 14import h2.config
 15import h2.connection
 16import re
 17import socket
 18
 19
 20def establish_tcp_connection():
 21    """
 22    This function establishes a server-side TCP connection. How it works isn't
 23    very important to this example.
 24    """
 25    bind_socket = socket.socket()
 26    bind_socket.bind(('', 443))
 27    bind_socket.listen(5)
 28    return bind_socket.accept()[0]
 29
 30
 31def receive_initial_request(connection):
 32    """
 33    We're going to receive a request. For the sake of this example, we're going
 34    to assume that the first request has no body. If it doesn't have the
 35    Upgrade: h2c header field and the HTTP2-Settings header field, we'll throw
 36    errors.
 37
 38    In production code, you should use a proper HTTP/1.1 parser and actually
 39    serve HTTP/1.1 requests!
 40
 41    Returns the value of the HTTP2-Settings header field.
 42    """
 43    data = b''
 44    while not data.endswith(b'\r\n\r\n'):
 45        data += connection.recv(8192)
 46
 47    match = re.search(b'Upgrade: h2c\r\n', data)
 48    if match is None:
 49        raise RuntimeError("HTTP/2 upgrade not requested!")
 50
 51    # We need to look for the HTTP2-Settings header field. Again, in production
 52    # code you shouldn't use regular expressions for this, but it's good enough
 53    # for the example.
 54    match = re.search(b'HTTP2-Settings: (\\S+)\r\n', data)
 55    if match is None:
 56        raise RuntimeError("HTTP2-Settings header field not present!")
 57
 58    return match.group(1)
 59
 60
 61def send_upgrade_response(connection):
 62    """
 63    This function writes the 101 Switching Protocols response.
 64    """
 65    response = (
 66        b"HTTP/1.1 101 Switching Protocols\r\n"
 67        b"Upgrade: h2c\r\n"
 68        b"\r\n"
 69    )
 70    connection.sendall(response)
 71
 72
 73def main():
 74    """
 75    The server upgrade flow.
 76    """
 77    # Step 1: Establish the TCP connection.
 78    connection = establish_tcp_connection()
 79
 80    # Step 2: Read the response. We expect this to request an upgrade.
 81    settings_header_value = receive_initial_request(connection)
 82
 83    # Step 3: Create a H2Connection object in server mode, and pass it the
 84    # value of the HTTP2-Settings header field.
 85    config = h2.config.H2Configuration(client_side=False)
 86    h2_connection = h2.connection.H2Connection(config=config)
 87    h2_connection.initiate_upgrade_connection(
 88        settings_header=settings_header_value
 89    )
 90
 91    # Step 4: Send the 101 Switching Protocols response.
 92    send_upgrade_response(connection)
 93
 94    # Step 5: Send pending HTTP/2 data.
 95    connection.sendall(h2_connection.data_to_send())
 96
 97    # At this point, you can enter your main loop. The first step has to be to
 98    # send the response to the initial HTTP/1.1 request you received on stream
 99    # 1.
100    main_loop()

Prior Knowledge

It’s possible that you as a client know that a particular server supports HTTP/2, and that you do not need to perform any of the negotiations described above. In that case, you may follow the steps in HTTPS URLs (ALPN), ignoring all references to ALPN: there’s no need to perform the upgrade dance described in HTTP URLs (Upgrade).