Skip to main content

HTTP Series - Part-1: (TCP, HTTP and HTTP/1.1)

Hypertext Transfer Protocol (HTTP) enables the transfer of data over the Internet. HTTP is an application-layer protocol that facilitates the transmission of hypermedia documents, such as HTML. It was designed for communication between web browsers and servers but can also be used for other purposes. I have discussed in detail about different modes of communication between a client and a server. They all use HTTP under the hood.

HTTP is a “stateless” protocol, which means each request is executed independently, without any knowledge of the requests that were executed before it. It uses the underlying transport protocol TCP (Transmission Control Protocol) to establish and manage connections between a client and a server. 

Transmission Control Protocol (TCP):

TCP is a connection-oriented transport layer protocol. It provides a fully duplex and reliable exchange of messages between different devices over a network. Some of the main features of TCP are:

  • Reliability:
    • TCP ensures the reliable delivery of data by implementing mechanisms such as acknowledgment (ACK) and retransmission.
    • When a sender transmits data, it expects an acknowledgment from the receiver. If an acknowledgment is not received within a certain timeframe, the sender assumes that the data was not successfully delivered and retransmits it.
  • Connection Oriented:
    • TCP establishes a connection between the communicating devices before data exchange begins.
  • Ordered Delivery:
    • TCP guarantees the ordered delivery of data. Each segment the sender sends is assigned a sequence number, and the receiver uses these sequence numbers to reorder the segments if they arrive out of order.
  • Full-Duplex Communication:
    • TCP supports full-duplex communication, meaning that data can be sent and received simultaneously between two devices. 
  • Flow Control:
    • TCP employs flow control mechanisms to prevent the sender from overwhelming the receiver with data. 
    • Both the sender and receiver negotiate during handshake (more on handshake in the next section) and maintain a window size, indicating the amount of data that can be sent before receiving an acknowledgment. 
    • As the receiver acknowledges segments, the window slides forward(Sliding window Protocol), indicating that more data can be sent. TCP also allows for dynamic adjustment of the window size based on network conditions.

Hypertext Transfer Protocol (HTTP):

HTTP follows a classical client-server model, with a client opening a connection to make a request, and then waiting until it receives a response. The server processes these requests and sends back corresponding HTTP responses. HTTP works on top of TCP. The process of creating an HTTP connection involves several steps:

1. TCP 3-Way Handshake:

  • The client initiates the connection by sending a TCP packet with the SYN flag (synchronize) set to the server. This packet includes an Initial Sequence Number (ISN) chosen by the client.
  • Upon receiving the SYN packet, the server acknowledges the request by sending a TCP packet with both the SYN and ACK flags set. The acknowledgment number (ACK) is set to the Client's ISN plus 1, and the server also chooses its ISN. 
  • The client responds with an ACK packet to acknowledge the Server's ISN. The acknowledgment number is set to the Server's ISN plus 1.
  • A small note: "ACK"  which is used as the flag, and the sequence number is different. There are 6 flags (aka Control bits) defined in RFC793 and the ACK flag is one of them. ACK sequence number (32-bit) is the value of the next sequence number the sender of the segment is expecting to receive.

2. Establishment of a TCP Connection:

  • Once the three-way handshake is complete, a TCP connection is established. The connection is identified by a unique combination of source IP address, source port, destination IP address, and destination port.

3. HTTP Request:

  • After the TCP connection is established, the client can send an HTTP request to the server. 
  • The HTTP request includes information such as the type of request (GET, POST, etc.), the requested resource (URI), headers, and any applicable data.

4. HTTP Response:

  • Upon receiving the HTTP request, the server processes the request, performs any necessary computations, retrieves data, and generates an HTTP response. 
  • The server sends the HTTP response back to the client over the established TCP connection.
  • The response includes status information, headers, and the requested data or indicates an error if applicable.

5. TCP Connection Termination:

  • Depending on the HTTP version and the headers exchanged during the connection, the connection may be closed immediately after the request and response (non-persistent connection) or kept open for potential future requests (persistent connection).
  • When the communication is complete, the TCP connection is terminated. This is typically done through a four-way handshake, where both the client and server exchange FIN (finish) and ACK flags to close the connection gracefully.
  • 4 Way Handshake for TCP connection Termination
    • The client initiates the termination by sending a TCP packet with the FIN flag (finish) set to the server. This indicates that the client has no more data to send.
    • The server acknowledges the client's FIN with an ACK packet. At this point, the server may still have data to send to the client. The acknowledgment number (ACK) is set to the client's sequence number plus 1.
    • Once the server is ready to terminate the connection, it sends a TCP packet with the FIN flag set to the client. This indicates that the server has no more data to send.
    • The client acknowledges the server's FIN, indicating that it received the termination request. The acknowledgment number is set to the server's sequence number plus 1. After this acknowledgment, the client and server can independently close their respective sides of the connection.

    • Curious minds can dive deep into how the state changes during TCP termination: here
It's worth emphasizing that within the outlined 5 steps, the initial and final stages impose a significant resource burden. It is resource-intensive to create and close the TCP connections. HTTP/1.0 establishes and terminates the entire TCP connection for each HTTP request. HTTP/1.1 came with an optimization. A single TCP connection can remain open, allowing the client and server to exchange multiple HTTP requests and responses efficiently within that same connection. Since the connection is persisted across HTTP requests/responses, HTTP/1.1 is a persistent connection whereas HTTP/1.0 is non-persistent.

HTTP/1.1 Key Features:

  • Persistent Connection:
    • One of the significant improvements in HTTP/1.1 is the support for persistent connections. Unlike HTTP/1.0, which opens a new connection for each request, HTTP/1.1 allows multiple requests and responses to be sent over the same TCP connection, reducing latency.
  • Connection Management:
    • HTTP/1.1 includes features like connection reuse (Connection: keep-alive), allowing multiple requests to be sent over a single connection. This minimizes the overhead of opening and closing connections for each request.
  • Content Compression:
    • HTTP/1.1 introduces support for content compression using mechanisms like gzip. This helps reduce the amount of data transferred over the network, improving performance.
    • The client sends the request header  Accept-Encoding: gzip, deflate, and the server responds with the response header Content-Encoding: gzip if compression is applied. 
  • Chunked Transfer-Encoding:
    • The header Transfer-Encoding: chunked enables the server to send the response in smaller, variable-sized chunks. This is useful for streaming content or for situations where the total size of the response is unknown. e.g: Server-Sent Events
  • Upgrade Header:
    • The Upgrade header allows clients and servers to negotiate protocol upgrades, enabling the transition to newer protocols like WebSocket.

HTTP/1.1 has its own set of limitations, including inefficiencies arising from multiplexing requests over a single connection, head-of-line blocking, repetitions of header data, and more. In the next part, we will delve into these limitations and explore how they were addressed in the subsequent version, HTTP/2.

Comments

Post a Comment

Popular posts from this blog

CAP Theorem - Debunking Myths

The CAP theorem is a widely recognized idea in the field of distributed systems. It represents three key concepts: Consistency, Availability, and Partition Tolerance. While most of us are familiar with its definition, the devil lies in the details. In this discussion, we'll clarify common myths and misunderstandings. We'll start by explaining the CAP theorem in detail, and then explore various scenarios that may challenge the common interpretation of it. CAP theorem also known as Brewer's theorem states that any distributed data store can provide only two of the following three guarantees: Consistency:  For every read request, the system should provide the most recent write or an error. Note that this consistency is different from the consistency of the  ACID theorem Availability:   For every request, the system should provide a response, even if it’s not the latest data.  In other words, all non-failing (healthy) nodes in the distributed system return a valid ...
Post a Comment
Read more

Understanding Merkle Tree

A Merkle Tree is a cryptographic tree structure used in computer science and distributed systems to efficiently verify the integrity of large sets of data (accuracy and consistency of data over its lifecycle).  Merkle Tree, also known as Hash Tree is a tree of hash values.  It has a tree structure in which each leaf node is a hash of a small portion of the data, and each non-leaf node is a hash of its children. It is used in applications such as  NoSQL databases, Git, Cryptocurrencies,  File Systems, etc. Some key characteristics of Merkle Tree are: Binary Tree Structure:  The Merkle Tree is a binary tree, where each leaf node represents a hash of data. Leaf Nodes: The data set is divided into fixed-size blocks or "leaves". Each leaf node contains the hash of a specific data block or piece of information. Non-Leaf Nodes: Non-leaf nodes in the tree represent the hash of the concatenation of their child node's hashes.  If the number of leaves is odd...
Post a Comment
Read more

Event Driven Architecture - SAGA Pattern (Part-1 : Choreography Model)

The Saga pattern is a distributed transactional pattern used in microservices architecture to maintain data consistency across multiple services. It helps manage long-running transactions involving multiple services by breaking them down into smaller, more manageable work units. There is a famous Database per Service  pattern in the Microservice world. Under this paradigm, each service maintains its own dedicated database. Some business transactions, however, span multiple services so we need a mechanism to implement transactions that span through services. Take, for instance, the scenario of placing an online order, which involves actions like inventory verification and item reservation till payment completion. Since services such as Orders, Inventory, and Payment operate on separate databases, the application cannot simply use a local ACID transaction. 2 Phase Commit Protocol  is one of the options being used for ensuring transactions across services. However, it has se...
Post a Comment
Read more