Software engineering notes

Network

OSI Model

The OSI model is a theoretical model designed to understand and describe how different network protocols interact and work together. In the real world, protocols don’t always fit neatly into a single layer and may operate across multiple layers.

Standard OSI model

  1. Layer 1 - Physical
    • electric signals / driver
    • the data gets converted into a bit stream (1/0)
    • Bits -> Signal
  2. Layer 2 - Data Link
    • frames / mac address / ethernet
    • facilitates data transfer between two devices on the same network
    • Frame (breaks packets into smaller pieces called frames)
  3. Layer 3 - Network
    • IP / ARP / ICMP
    • destination and source IP
    • Packet (break segments into smaller units called packets and reassembles these packets on the receiving device)
  4. Layer 4 - Transport
    • TCP / UDP
    • destination and source Port to identify services or applications
    • Segment (break the data into chunks called segments)
  5. Layer 5 - Session
    • connection establishment / TLS
    • state / stateful / cookie session
  6. Layer 6 - Presentation
    • encoding / serialisation
    • JSON object / UTF-8
  7. Layer 7 - Application
    • HTTP / FTP / gRPC

TCP/IP model

  1. Network Access (Data Link + Pyhsical)
  2. Internetwork (Network)
  3. Transport (Transport)
  4. Application (Session + Presentation + Application)

An example to explain how OSI model deal with sending a POST request to HTTPs endpoint

Ref:

Heartbeat packet

A heartbeat packet is a type of signal sent at regular intervals between two systems to check that the connection is still alive and functioning properly

It’s used to check the health of servers

Heartbeat doesn’t belong to a specific OSI models. It’s more of an operational concept that can be implemented at different layers.

Load Balancer

IP

Basics

Subnet mask

Classes

Special IP address ranges that are used for special purposes are:

Proxy server vs Reverse proxy server

regular proxy server

flow

client -> proxy server -> server

For example: CDN and Nginx

reverse proxy server

flow

client -> reverse proxy server
                    |--------------> server A
                    |--------------> server B
                    |--------------> server C

For example: Load balancer

The difference between regular proxy server and reverse proxy server

reverse proxy server needs a set of rules for forwarding requests to the destination server while regular proxy server simply forwards the request without any specific rules

P2P connection

Introduction

When two servers behind their own NATs need to establish a P2P connection, they need to exchange information about their private IP and port numbers to establish a direct connection. This process is called NAT traversal.

Techniques for NAT Traversal

The steps to establish a p2p connection between 2 servers behind their own NATs

The first server sends a connection request to the second server. The connection request includes the first server’s private IP and port as well as the public IP and port of the NAT device that the first server is behind. The second server receives the connection request from the first server, but because it is behind a NAT device, the request appears to come from the public IP and port of the NAT device and not the private IP and port of the first server.

                                SDP offer
First Server (FS)   ─────>>>───[Connection Request]───────>>>   Second Server (SS)
                               Private IP & Port (FS)
                               Public IP & Port (NAT FS)

The second server sends a response back to the first server. This response includes the second server’s private IP and port as well as the public IP and port of the NAT device that the second server is behind. The first server receives the response from the second server, but because it is behind a NAT device, the response appears to come from the public IP and port of the NAT device and not the private IP and port of the second server.

                                    SDP answer
First Server (FS)   <<<─────────────[Response]──────<<<──────   Second Server (SS)
                                  Private IP & Port (SS)
                                  Public IP & Port (NAT SS)

The first server sends a second connection request to the second server. This request includes the private IP and port of the first server and the public IP and port of the NAT device that the first server is behind, as well as the private IP and port of the second server and the public IP and port of the NAT device that the second server is behind.

First Server (FS)   ───>>>───[2nd Connection Request]─────>>>   Second Server (SS)
                                  Private IP & Port (FS)
                                  Public IP & Port (NAT FS)
                                  Private IP & Port (SS)
                                  Public IP & Port (NAT SS)

The second server receives the second connection request from the first server, and because it now has both private and public IP and port information for both servers, it is able to create a mapping in its NAT device that allows incoming traffic from the first server to be routed to the second server.

First Server (FS)   ───> NAT ────[2nd Connection Request]───> NAT ───> Second Server (SS)
                          ├─ Private IP & Port (FS)            ├─ Private IP & Port (FS)
                          ├─ Public IP & Port (NAT FS)         ├─ Public IP & Port (NAT FS)
                          ├─ Private IP & Port (SS)            ├─ Private IP & Port (SS)
                          └─ Public IP & Port (NAT SS)         └─ Public IP & Port (NAT SS)

The two servers are now able to establish a P2P connection and communicate directly with each other.

First Server (FS)    <────────────[P2P Connection]──────────>   Second Server (SS)

TCP

3-way handshake

diagram

Client                                                                Server
  |                                                                     |
  |----- SYN (Sequence Number: X) ------------------------------------->|
  |                                                                     |
  |<---- SYN-ACK (Sequence Number: Y, Acknowledgment Number: X + 1) ----|
  |                                                                     |
  |----- ACK (Acknowledgment Number: Y + 1) --------------------------->|
  |                                                                     |

TCP states

TIME_WAIT

When a connection is closed, it will be turned into the state TIME_WAIT. Network in real world can be unpredicable. There is no gurantee for packet to be delivered in order. When lost packet was delayed to ask to close the connection, which was already disconnected, server might be confused by it and close the new established connection if it doesn’t have TIME_WAIT state. TIME_WAIT keeps the info of source address, source port, destination address and destination port. When server recevied delayed packet to ask for closing connection, server can use TIME_WAIT to verify whether it has been closed or not. TIME_WAIT’s purpose is for preventing server from messing up the new connection from the same address.

When you close the socket’s file descriptor, the file descriptor itself is closed, but the socket in TIME_WAIT will still consume file descriptors. Therefore, a socket in TIME_WAIT consumes file descriptors

查看系統 TCP 相關設定

MacOS:

sysctl net.inet.tcp

simultaneously maximum connection for a server

ref: https://www.quora.com/What-is-the-maximum-number-of-concurrent-tcp-connections-system-can-support

The theoretical maximum number of connections per client per server port is 65534.
Assuming one network interface (i.e., 1 IP on your host),
then you could potentially make ~4 million (IP address space size) x 65534.
I think maybe we should talk in logarithmic terms since this number is already so huge :)
What will bite you before that will be other issues - such as the design of application which is making or handling such connections,
your OS's TCP/IP stack design or ultimately the amount of memory.
If memory is not the issue and you want to increase that number,
you can add another network interface and double this number further (ie., increase number of clients).

ref: https://medium.com/fantageek/understanding-socket-and-port-in-tcp-2213dc2e9b0c

What is the maximum number of concurrent TCP connections that a server can handle, in theory ?

A single listening port can accept more than one connection simultaneously.
There is a ‘64K’ limit that is often cited, but that is per client per server port, and needs clarifying.
If a client has many connections to the same port on the same destination,
then three of those fields will be the same — only source_port varies to differentiate the different connections.
Ports are 16-bit numbers, therefore the maximum number of connections any given client can have to any given host port is 64K.
However, multiple clients can each have up to 64K connections to some server’s port,
and if the server has multiple ports or either is multi-homed then you can multiply that further
So the real limit is file descriptors. Each individual socket connection is given a file descriptor,
so the limit is really the number of file descriptors that the system has been configured to allow and resources to handle.
The maximum limit is typically up over 300K, but is configurable e.g. with sysctl

nf_conntrack: table full, dropping packet

What’s nf_conntrack?

It is a feature to allow kernel to keep track of connections. It might be only used in haproxy server. When NAT or firewall works, it’s nf_conntrack under the hood. nf_conntract records connections info, including the mapping between public IPs (external IPs) and private IPs (internal IPs) for NAT, so that the packets can be sent to the right end.

What are common causes?
How to solve this?

Note: If it is full, it won’t accept new connection. There are 3 ways to fix it:

If nf_conntrack is disabled, what would be the impact?

VPN

讓 Private 可以被特定的連線操作

Unix Domain Socket, aka IPC socket

Unix Domain Socket is a data communications endpoint for exchanging data between processes executing on the same host operating system. It supports transmission of a reliable stream of bytes, ordered and reliable transmission of datagrams. The API for Unix domain sockets is similar to that of an Internet socket, but rather than using an underlying network protocol, all communication occurs entirely within the operating system kernel. Unix domain sockets use the file system as their address name space. Processes reference Unix domain sockets as file system inodes, so two processes can communicate by opening the same socket. Instead of identifying a server by an IP address and port, a Unix domain socket is known by a pathname. Obviously the client and server have to agree on the pathname for them to find each other.

RPC

Remote procedure call (RPC) is an Inter-process communication technology that allows a computer program to cause a subroutine or procedure to execute in another address space (commonly on another computer on a shared network) without the programmer explicitly coding the details for this remote interaction.

An RPC (remote procedure call) is a form of IPC (inter-process communication)

gRPC

RPC vs IPC

Named Pipe vs IPC

ref:

traceroute 觀察 host 經過的節點狀態

$ traceroute google.com
traceroute to google.com (172.217.31.142), 30 hops max, 60 byte packets
 1  ec2-175-41-192-150.ap-northeast-1.compute.amazonaws.com (175.41.192.150)  16.685 ms ec2-175-41-192-144.ap-northeast-1.compute.amazonaws.com (175.41.192.144)  19.225 ms ec2-175-41-192-146.ap-northeast-1.compute.amazonaws.com (175.41.192.146)  16.309 ms
 2  100.64.1.200 (100.64.1.200)  17.298 ms 100.64.3.78 (100.64.3.78)  13.307 ms 100.64.0.78 (100.64.0.78)  21.296 ms
 3  100.66.3.36 (100.66.3.36)  17.270 ms 100.66.3.108 (100.66.3.108)  20.964 ms 100.66.3.192 (100.66.3.192)  14.131 ms
 (...略...)
16  108.170.242.193 (108.170.242.193)  4.186 ms 108.170.242.161 (108.170.242.161)  3.221 ms 108.170.242.193 (108.170.242.193)  5.237 ms
17  74.125.251.237 (74.125.251.237)  3.611 ms  3.618 ms  5.141 ms
18  nrt20s08-in-f14.1e100.net (172.217.31.142)  2.942 ms  4.001 ms  2.928 ms