Blog/Random notes about network programming

So I'm trying to get a deeper better and deeper understanding of network programming in the Unix environment, and in this post I'll collect various bits of information that I've found on-line, as well as the results of some digging here (that, is the results of touring the rabbit holes).

As (so far) I owe most of my learnings on the subject to the Beej's Guide to Network Programming you might recognize some referencing to that content. Mostly, I'm going to write additional notes ("deep dives") on stuff presented in there.

EDIT: In this first page I've collected a lot of progress, and it's grown up quite a bit. I think I'll log further progress in other pages, possibly in their own category, so that we have "episodes".

Disclaimer

Needless to say, this page will be a perennial work in progress. As it is somehow a reflection of the current state of my understanding of the subject do not expect this page to be 100% correct.

Also, I do all of my work (and my hobby) programming in GNU/Linux systems, compatibility with other UNIX systems (BSDs etc) is not an objective in any way. I might occasionally include compatibility stuff with Mac OS, mostly because I've been given a Macbook Pro for work.

This page might also contain discussion and reference to tangentially related topics.

Pre-requisites for exploration

Having worked with other languages and runtimes I'm quite used to having something like a library reference where I can find all of the stuff (libraries/packages, classes, functions, constants, enums etc) available, ideally in some ordered/indexed manner.

Some useful resources on the matter I've found so far:

• C headers reference from cppreference.com
• the Single Unix Specification ("SUS" from here on) contains somewhat of an indexed reference of all the things you can expect to find in a POSIX-compliant system. It comes in four "volumes": base definition (XBD), system interfaces (XSH), shell utilities (XCU) and rationale (XRAT). It's available for download too (having an offline copy is a good idea)
  • On-line version: https://pubs.opengroup.org/onlinepubs/9699919799/
• Some additional clarifications here and there have been looked up from "Advanced Programming in the Unix Environment" (3rd edition by Stevens & Rago)
  • This book so far has been less useful than it seemed before buying it. It's great for consultation purposes after you've actually learned the topics, but I wouldn't recommend it much to a self-learner.

It goes without saying that you'll need a linux system, a decent compiler and a decent c library. I'm doing most of my tests either on a RHEL7-like system or on a Fedora system and in both cases I'm using gcc as a compiler and the glibc as a c library.

The man-pages-posix package contains documentation taken from SUS. It can be very useful to have pages from the SUS handy in a terminal. To see what man pages are available on my system it was sufficient to type ls /usr/share/man/man0p.

The getaddrinfo function

The getaddrinfo (from netdb.h ) is the main utility needed when gathering the necessary data to instantiate a socket.

According to SUS (bold emphasis mine)

«The getaddrinfo() function shall translate the name of a service location (for example, a host name) and/or a service name and shall return a set of socket addresses and associated information to be used in creating a socket with which to address the specified service.»

Essentially it does address resolution (dns name to ip address, in the appropriate C structures) but it can also be use to prepare the data for server sockets (that is, sockets that we intend to call bind on). This C code shows the simplest way to perform a dns resolution (and nothing else, not even printing):

#include <stdio.h>
#include <stdlib.h>

#include <netdb.h> // for getaddrinfo, gai_strerr and struct addrinfo

/* int getaddrinfo(const char *node,
                   const char *service,
                   const struct addrinfo *hints,
                   struct addrinfo **res);
*/


int main(int argc, char **argv) {
  int gai_res ;
  struct addrinfo *res ;
  struct addrinfo *res_p ;

  gai_res = getaddrinfo("www.google.com", "http", NULL, &res); // works
  //gai_res = getaddrinfo("www.google.it", NULL, NULL, &res); // works
  //gai_res = getaddrinfo(NULL, "http", NULL, &res); // works (but why?)
  //gai_res = getaddrinfo(NULL, NULL, NULL, &res); // fails

  if (gai_res) {
    printf("%s\n", gai_strerror(gai_res)) ;
    exit(1);
  } 
  printf("getaddrinfo: OK\n");

  res_p = res->ai_next;
  do {
    printf("Got a result\n");
    res_p = res_p->ai_next ;
  } while ( res_p != NULL) ;

  freeaddrinfo(res);
  freeaddrinfo(res_p) ;
  
  exit(0);
}

The node and service argument are respectively the hostname (eg: "www.amazon.com") and the port to connect to. The service argument is a string as it is custom to have an /etc/services file on the system mapping common services to port numbers. Here is an excerpt of /etc/services from a Linux machine:

lmtp            24/tcp                          # LMTP Mail Delivery
lmtp            24/udp                          # LMTP Mail Delivery
smtp            25/tcp          mail
smtp            25/udp          mail
time            37/tcp          timserver
time            37/udp          timserver
rlp             39/tcp          resource        # resource location
rlp             39/udp          resource        # resource location
nameserver      42/tcp          name            # IEN 116
nameserver      42/udp          name            # IEN 116
re-mail-ck      50/tcp                          # Remote Mail Checking Protocol
re-mail-ck      50/udp                          # Remote Mail Checking Protocol
domain          53/tcp                          # name-domain server
domain          53/udp

If a mapping for the port we need does not exist, then the port can be expressed as a string (eg: port 8080 would be expressed as "8080")

Interestingly at line 20 of the C code example we can see that getaddrinfo(NULL, "http", NULL, &res); works, but i don't (yet) know why. I suspect that kind of usage might be useful when gathering data for server sockets. Later I'll dig deeper in the kind of results obtained by that call. On the machines where I'm executing that code I get six "Got a result" lines and I have three network interfaces with two addresses (one ipv4, one ipv6) each, so my guess is that's returning getting the addresses for those.

Two main parameters are worth mentioning:

• hints
• res

res is fairly simple: it's the result of the function call. It is a pointer to a linked list of type struct addrinfo terminated by a null pointer.

hints on the other hand, is way more complicated and contains the "parameters" for the host name to address resolution.

Interestingly, both res and hints are of the same type: struct addrinfo. More on this later

The addrinfo structure

One of the annoyances I felt when learning about this struct boils down to, essentially, these questions:

1. What are the values I can use for the various fieds, and where do I find all the possible such values?
2. How do I fill this structure when performing address resolution?
3. How do I use the fields when i have performed address resolution?

The following fields are defined in struct addrinfo:

• int ai_flags: Input flags.
  • more on this later - but all the values you need are in netdb.h
• int ai_family: Address family of socket.
  • this where you ask for an ipv4, ipv6, or UNIX socket (respectively: AF_INET , AF_INET6 or AF_UNIX; all defined in sys/socket.h -- but you can also use AF_UNSPEC if you don't care whether the result is IPv4 or IPV6)
• int ai_socktype: Socket type
  • this is usually SOCK_STREAM or SOCK_DGRAM or others defined in sys/socket.h
• int ai_protocol: Protocol of socket.
  • this could be stuff like IPPROTO_TCP or IPPROTO_UDP (from netinet/in.h , or from /etc/protocols via some querying - see getprotobyname and getprotobynumber in netdb.h)
• socklen_t ai_addrlen: Length of socket address
  • important as length of ai_addr differs between, say, ipv4 and ipv6
• struct sockaddr *ai_addr: Socket address of socket.
  • you can expect getaddrinfo to fill this
  • this field contains the actual result of the dns name resolution
  • most likely, you're going to cast this to sockaddr_in or sockaddr_in6 to extract, respectively, the ipv4 or ipv6 address
  • you can look at the ai_family field to determine if you need to cast to sockaddr_in or sockaddr_in6
• char *ai_canonname: Canonical name of service location
  • you can expect getaddrinfo to fill this
• struct addrinfo *ai_next  Pointer to next in list
  • you can expect getaddrinfo to fill this when there is more than one result (null-pointer otherwise)

The ai_flags field

This field is meaningful when filled in the hints parameter, and meaningless when read from the results (from the res parameter).

In the netdb.h reference page the following constants are mandated for inclusion, meant to be used in the ai_flags field of the addrinfo struct:

• AI_PASSIVE
  • The results will later be used with bind. This is usually the case when looking up information and preparing sockaddr_in structs for server sockets.
• AI_CANONNAME
  • Request the canonical name of the host. SUS doesn't say much of what is considered the canonical name.
  • The man7.org says the following: «If hints.ai_flags includes the AI_CANONNAME flag, then the ai_canonname field of the first of the addrinfo structures in the returned list is set to point to the official name of the host.»
  • Two StackOverflow posts seems to clarify the topic:
    • What can be expected in the field ai_canonname when calling getaddrinfo
    • getaddrinfo, I am not getting any canonname
• AI_NUMERICHOST
  • When this flag is set, the node must already be a numeric address (a string containing an ipv4, for example)
  • This flag avoids doing network host address lookups. Essentially, this skips dns.
• AI_NUMERICSERV
  • When this flag is set, the service must be not null and must be a numeric port number.
  • This flag avoids looking up the service in /etc/services and other places.
• AI_V4MAPPED
  • «If no IPv6 addresses are found, query for IPv4 addresses and return them to the caller as IPv4-mapped IPv6 addresses.»
• AI_ALL
  • Query both ipv4 and ipv6 addresses
• AI_ADDRCONFIG
  • «Query for IPv4 addresses only when an IPv4 address is configured; query for IPv6 addresses only when an IPv6 address is configured.»

This field is the kind of place where the SUS might require something basic but the actual implementation might do more. Hence, the man-page (web version ) from Linux should also be consulted when working on Linux systems.

The sockaddr struct and related structs

The whole thing with struct sockaddr is a bit messy. From my understanding, the messy-ness comes from two fronts:

• the goal of having a single interface (struct sockaddr) for ip(v4) sockets and unix sockets
• an effort to keep backwards-compatibility when introducing ipv6 sockets.

So long story short:

• you usually work with protocol-specific structs (these would be struct sockaddr_in, struct sockaddr_in6, struct sockaddr_un)
• you cast protocol-specific structs (the ones mentioned in the point above) to the generic struct sockaddr when passing such data to function (eg: bind).

All of the involved structures (struct sockaddr, struct sockaddr_in/in6) have sa_family field which which containes the address family (AF_INET/AF_INET6,AF_UNIX etc) which helps the receiving function treat the data correctly.

The struct sockaddr_storage thingy

Long story short, a struct sockaddr_storage is a struct large enough to hold any of the various struct sockaddr or struct sockaddr_* types (it's also padded and aligned in a way that doesn't mess up stuff, but you can largely ignore this aspect).

SUS says that whenever a struct sockaddr_storage is casted to struct sockaddr, the ss_family field must map to the sa_family field, and that when casted to any of the struct sockaddr_in/in6/un then again, the ss_family field must map to the sa_field.

Quoting from the SUS page for sys/socket.h (bold emphasis not mine)

When a pointer to a sockaddr_storage structure is cast as a pointer to a sockaddr structure, the ss_family field of the sockaddr_storage structure shall map onto the sa_family field of the sockaddr structure. When a pointer to a sockaddr_storage structure is cast as a pointer to a protocol-specific address structure, the ss_family field shall map onto a field of that structure that is of type sa_family_t and that identifies the protocol's address family.

The idea (again: if my understanding is correct) it to provide a correct/supported/standard way of allocating a memory area capable of hosting any of the other structures. If you don't know in advance what kind of address family you'll be working with... Allocate a struct sockaddr_storage and cast later (when you get to know) to whatever you need.

More on struct sockaddr and its relatives

So far, useful references on the matter can be found at:

• A description of the sockaddr (and related structures) can be found in the description of sys/socket.h
• The manpage for sockaddr (man-pages project, section 3type)
  • This also contains
• The manpage from the linux kernel about the ip protocol implementation: man 7 ip
• The SUS reference page for netinet/in.h also contains description of the sockaddr_in, sockaddr_in6 structures

Contents of the sockaddr structures...

... finally.

Long story short: you most often get a struct sockaddr, but can't use that directly. In order to be able to access the data in a meaningfully way you must cast that to a protocol specific structure (most often sockaddr_in or sockaddr_in6).

A sockaddr_in structure has (at least) three fields:

• sa_family_t sin_family
  • This is of fixed value of AF_INET.
• in_port_t sin_port
  • Port number.
• struct in_addr  sin_addr
  • This is a structure where the ip address is hold.
  • It has (to spec) only a single field, s_addr, of type in_addr_t.

A weird quirk of the in_addr struct

Reading some more on the matter, I've noticed that the the definition for struct in_addr is the following:

struct in_addr {
  in_addr_t s_addr;
}

The focus here is on the fact that such structure has only one field (at least according to SUS spec).

Knowing how structs work in the C language, this means that for some (most?) purposes the structs of type in_addr and their only field within, s_addr, are interchangeable. However, swapping the field for the whole struct is probably a bad idea

From a quick test, the following two pieces of code appear to produce the same output:

Original piece of code:

if ( (inet_ntop(AF_INET,
                &(sa_v4->sin_addr.s_addr), // <-- this is correct
                (void*)&ipv4_result_string,
                INET_ADDRSTRLEN)) != NULL){
  printf("Ipv4 address: %s\n", ipv4_result_string);
}

Swapped the field for the whole struct:

if ( (inet_ntop(AF_INET,
                &(sa_v4->sin_addr), // <-- change is here
                (void*)&ipv4_result_string,
                INET_ADDRSTRLEN)) != NULL){
  printf("Ipv4 address: %s\n", ipv4_result_string);
}

The two lines of code appear to produce the same output.

It'd be interesting to see what sizeof returns for struct in_addr on various systems/platforms. sizeof(struct in_addr) appears to be 4, at least on Linux running on x86_64.

Putting it all together: looking up ip addresses for an hostname

The code below uses getaddrinfo to lookup ip addresses (both ipv4 and ipv6) for www.amazon.com and prints them to standard out.

#include <stdio.h>
#include <stdlib.h>

#include <netdb.h>
#include <arpa/inet.h>
#include <sys/socket.h>


int main(int argc, char **argv) {
  int gai_res ;
  struct addrinfo *res ;
  struct addrinfo *res_p ;

  char *target = "en.wikipedia.org";
  printf("Looking up: %s\n", target);

  gai_res = getaddrinfo(target, NULL, NULL, &res);

  if (gai_res) {
    printf("%s\n", gai_strerror(gai_res)) ;
    exit(1);
  }
  // printf("getaddrinfo: OK\n");

  struct sockaddr_in  *sa_v4;
  char ipv4_result_string[INET_ADDRSTRLEN];

  struct sockaddr_in6 *sa_v6;
  char ipv6_result_string[INET6_ADDRSTRLEN];

  res_p = res->ai_next;

  do {
    if (res_p->ai_family == AF_INET) {
      sa_v4 = (struct sockaddr_in*)res_p->ai_addr;
      if ( (inet_ntop(AF_INET,
                      &(sa_v4->sin_addr.s_addr),
                      (void*)&ipv4_result_string,
                      INET_ADDRSTRLEN)) != NULL){
        printf("Ipv4 address: %s\n", ipv4_result_string);
      } else {
        printf("Failed to convert ipv4 to string :(\n");
      }
    } else if (res_p->ai_family == AF_INET6) {
      sa_v6 = (struct sockaddr_in6*)res_p->ai_addr;
      if ( (inet_ntop(AF_INET6,
                      &(sa_v6->sin6_addr.s6_addr),
                      (void*)&ipv6_result_string,
                      INET6_ADDRSTRLEN)) != NULL){
        printf("Ipv6 address: %s\n", ipv6_result_string);
      } else {
        printf("Failed to convert ipv6 to string :(\n");
      }
    }
    res_p = res_p->ai_next ;
  } while ( res_p != NULL) ;

  freeaddrinfo(res);
  exit(0);
}

The only thing not previously discussed is the inet_ntop function. For that bit, the Beej's guide to network programming is sufficient.