Have you ever wanted to run a software stack locally, testing client/server
interactions, all on the same machine, under a normal user account?
Maybe cwrap could help.
What is cwrap?
cwrap is a set of tools to create a fully isolated network environment to
test client/server components on a single host.
It provides synthetic account information, hostname resolution and
support for privilege separation. The heart of cwrap consists of three
libraries you can preload to any executable.
cwrap consists of three libraries:
These libraries allow you to preload to a binary and so replacing library function calls,
by using LD_PRELOAD.
From the ld.so man pages:
- LD_PRELOAD
- A list of additional, user-specified, ELF shared libraries to
be loaded before all others. The items of the list can be
separated by spaces or colons. This can be used to
selectively override functions in other shared libraries.
I'm going to look at socket_wrapper and some examples of how to redirect
tcp sockets to use unix sockets.
Installation
I'm installing socket_wrapper-1.0.1 on Ubuntu 13.04.
The author of this software is Andreas Schneider, so before installing this I'm going to
import his pgp key.
PGP keys
simon@X220:~$ wget --quiet http://www.cryptomilk.org/0xCC014E3D-asn@cryptomilk.org-gpg_key.asc
simon@X220:~$ gpg --import 0xCC014E3D-asn@cryptomilk.org-gpg_key.asc
gpg: key CC014E3D: public key "Andreas Schneider <asn@cryptomilk.org>" imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
gpg: no ultimately trusted keys found
The project is part of the samba suite of software, so it's available for download
via the samba site.
Download
The latest version of socket_wrapper is socket_wrapper-1.0.1.tar.gz (04-Feb-2014 09:04, 36K)
simon@X220:~$ mkdir sockets
simon@X220:~$ cd sockets
simon@X220:~/sockets$ wget --quiet https://ftp.samba.org/pub/cwrap/socket_wrapper-1.0.1.tar.gz
simon@X220:~/sockets$ wget --quiet https://ftp.samba.org/pub/cwrap/socket_wrapper-1.0.1.tar.asc
simon@X220:~/sockets$ gzip -d socket_wrapper-1.0.1.tar.gz
simon@X220:~/sockets$ gpg --verify socket_wrapper-1.0.1.tar.asc
gpg: Signature made Tue 04 Feb 2014 16:04:36 GMT using RSA key ID CC014E3D
gpg: Good signature from "Andreas Schneider <asn@cryptomilk.org>"
gpg: aka "Andreas Schneider <asn@samba.org>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 8DFF 53E1 8F2A BC8D 8F3C 9223 7EE0 FC4D CC01 4E3D
simon@X220:~/sockets$ tar -xf socket_wrapper-1.0.1.tar
simon@X220:~/sockets$ cd socket_wrapper-1.0.1/
simon@X220:~/sockets/socket_wrapper-1.0.1$ mkdir builddir
simon@X220:~/sockets/socket_wrapper-1.0.1$ cd builddir/
Requirements
cmake is a requirement to build.
simon@X220:~/sockets/socket_wrapper-1.0.1/builddir$ sudo aptitude install cmake
Compile and Install
simon@X220:~/sockets/socket_wrapper-1.0.1/builddir$ cmake .. -DCMAKE_INSTALL_PREFIX:PATH=~/sockets/bin
-- The C compiler identification is GNU 4.7.3
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
......
-- Configuring done
-- Generating done
-- Build files have been written to: /home/simon/sockets/socket_wrapper-1.0.1/builddir
simon@X220:~/sockets/socket_wrapper-1.0.1/builddir$ make && make install
Scanning dependencies of target socket_wrapper
[100%] Building C object src/CMakeFiles/socket_wrapper.dir/socket_wrapper.c.o
Linking C shared library libsocket_wrapper.so
[100%] Built target socket_wrapper
[100%] Built target socket_wrapper
Install the project...
-- Install configuration: ""
-- Installing: /home/simon/sockets/bin/lib/pkgconfig/socket_wrapper.pc
-- Installing: /home/simon/sockets/bin/lib/cmake/socket_wrapper-config-version.cmake
-- Installing: /home/simon/sockets/bin/lib/cmake/socket_wrapper-config.cmake
-- Installing: /home/simon/sockets/bin/lib/libsocket_wrapper.so.0.0.1
-- Installing: /home/simon/sockets/bin/lib/libsocket_wrapper.so.0
-- Installing: /home/simon/sockets/bin/lib/libsocket_wrapper.so
Running netcat
In one terminal run netcat binding to IP: 127.0.0.10 on port 7
simon@X220:~/sockets/socket_wrapper-1.0.1/builddir$ cd ~/sockets/bin/
simon@X220:~/sockets/bin$ mktemp -d
/tmp/tmp.hxyAQf3Pda
simon@X220:~/sockets/bin$ LD_PRELOAD=lib/libsocket_wrapper.so \
> SOCKET_WRAPPER_DIR=/tmp/tmp.hxyAQf3Pda \
> SOCKET_WRAPPER_DEFAULT_IFACE=10 nc -v -l 127.0.0.10 7
Listening on [127.0.0.10] (family 0, port 7)
In a seperate terminal send a message to that IP and port.
simon@X220:~/sockets/bin$ LD_PRELOAD=lib/libsocket_wrapper.so \
> SOCKET_WRAPPER_DIR=/tmp/tmp.hxyAQf3Pda \
> SOCKET_WRAPPER_DEFAULT_IFACE=10 nc -v 127.0.0.10 7
Connection to 127.0.0.10 7 port [tcp/echo] succeeded!
Hello!
'Hello!' is then received by the listening netcat
simon@X220:~/sockets/bin$ LD_PRELOAD=lib/libsocket_wrapper.so \
> SOCKET_WRAPPER_DIR=/tmp/tmp.hxyAQf3Pda \
> SOCKET_WRAPPER_DEFAULT_IFACE=10 nc -v -l 127.0.0.10 7
Listening on [127.0.0.10] (family 0, port 7)
Connection from [127.0.0.10] port 7 [tcp/echo] accepted (family 2, sport 41505)
Hello!
Instead of netcat using tcp sockets, SOCKET_WRAPPER_DIR contains the unix sockets created in place of those tcp sockets.
simon@X220:~/sockets/bin$ file /tmp/tmp.hxyAQf3Pda/* && lsof /tmp/tmp.hxyAQf3Pda/*
/tmp/tmp.hxyAQf3Pda/T0A0007: socket
/tmp/tmp.hxyAQf3Pda/T0AA221: socket
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
nc 31334 simon 5u unix 0x0000000000000000 0t0 2265223 /tmp/tmp.hxyAQf3Pda/T0A0007
nc 31505 simon 3u unix 0x0000000000000000 0t0 2267187 /tmp/tmp.hxyAQf3Pda/T0AA221
Running Nginx
I created the most basic of nginx confs as a simple example:
simon@X220:~$ mkdir -p nginx-local/www
simon@X220:~$ touch nginx-local/www/test
simon@X220:~$ cd nginx-local
simon@X220:~/nginx-local$ vi nginx.conf
error_log /home/simon/nginx-local/error.log;
worker_processes 1;
events {
worker_connections 3;
}
http {
server {
listen 127.0.0.10:7;
root /home/simon/nginx-local/www;
location / {
autoindex on;
access_log off;
}
}
}
simon@X220:~/nginx-local$ LD_PRELOAD=/home/simon/sockets/bin/lib/libsocket_wrapper.so \
> SOCKET_WRAPPER_DIR=/tmp/tmp.hxyAQf3Pda \
> SOCKET_WRAPPER_DEFAULT_IFACE=10 \
> /usr/sbin/nginx -c /home/simon/nginx-local/nginx.conf \
> -g "pid /home/simon/nginx-local/nginx.pid;"
Note: You may receive an [alert] warning from nginx, that is cannot read the default error_log.
As of version 0.7.53, nginx will use the compiled-in default error log before, reading in
the config file.
Next, check that Nginx is running
simon@X220:~/nginx-local$ netstat -nlp | grep $(cat ~/nginx-local/nginx.pid)
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
unix 2 [ ACC ] STREAM LISTENING 2266669 919/nginx.pid; /tmp/tmp.hxyAQf3Pda/T0A0007
Connect as a client to the server and capture conversation (SOCKET_WRAPPER_PCAP_FILE)
simon@X220:~/nginx-local$ LD_PRELOAD=/home/simon/sockets/bin/lib/libsocket_wrapper.so \
> SOCKET_WRAPPER_DIR=/tmp/tmp.hxyAQf3Pda \
> SOCKET_WRAPPER_PCAP_FILE=/tmp/nginx.pcap \
> SOCKET_WRAPPER_DEFAULT_IFACE=10 telnet 127.0.0.10 7
Trying 127.0.0.10...
Connected to 127.0.0.10.
Escape character is '^]'.
GET / HTTP/1.1
Host: 127.0.0.10
HTTP/1.1 200 OK
Server: nginx/1.2.6 (Ubuntu)
Date: Thu, 01 May 2014 15:26:31 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
104
<html>
<head><title>Index of /</title></head>
<body bgcolor="white">
<h1>Index of /</h1><hr><pre><a href="../">../</a>
<a href="test">test</a> 01-May-2014 14:10 0
</pre><hr></body>
</html>
0
simon@X220:~/nginx-local$ kill -QUIT $(cat /home/simon/nginx-local/nginx.pid)
All of this network conversation is caught in the pcap file and so can be loaded into
wireshark.
simon@X220:~/nginx-local$ wireshark /tmp/nginx.pcap
Conclusion
This throws up a whole host of possibilities to test client/server interactions
that are required for your application and all tested locally as a normal
(non root) user.
The ability to easily translate IPv4 and IPv6 addresses to a unix sockets,
and simulate binding to privileged ports (< 1024), allows for
easier local testing, if your application requires a lot of network interactions
as part of it's test suite.
You may also be interested in the other components of cwrap
nss_wrapper
- Provides information for user and group accounts
- Network name resolution using a hosts file
Name resolution could work well with socket_wrapper and works much the same
way.
simon@X220:~/nss_wrapper$ echo "127.0.0.10 simonsblog.co.uk" > hosts
simon@X220:~/nss_wrapper$ LD_PRELOAD="bin/lib/libnss_wrapper.so" \
> NSS_WRAPPER_HOSTS=hosts \
> python -c "import socket; print(socket.gethostbyname('simonsblog.co.uk'))"
127.0.0.10
uid_wrapper
- Allows uid switching as a normal user.
- Start any application making it believe it is running as root.
- Support for user/group changing in the local thread using the syscalls (like glibc).
Check out the article on lwn.net and cwrap for more info.
There are comments.