Skip to main content

Get diff changes between original files installed with apt and current files

Vous avez modifié vos fichiers de conf et voulez comparer avec la version d'origine ? Vous auriez dû versionner vos fichiers de conf. :D Ou alors, ce petit script peut vous aider :

pkgdiff.sh :

#!/bin/sh
set -e

package=$1
tmp=$(mktemp -d --suffix=$package-pkgdiff)
echo "(Downloading package in $tmp)"
cd $tmp

apt-get download $package
ar vx ${package}*.deb
tar xJf data.tar.xz

find etc -type f |
while read file ; do
diff $file /$file
done

Usage : pkgdiff <pkg-name>

Pour juste avoir la liste des fichiers dans un packet : dpkg-query -L <pkg-name>

Déçu

4 min read

Envoyé à la liste MozFr en réponse à l'annonce d'un débat public sur la Loi Renseignement.

 

Déçu.

C'est en un mot mon point de vue sur le débat d'hier soir.

Je ne blâme pas les intervenants, bien au contraire. Leur diversité et leurs différents points de vue sur la question étaient intéressants et éclairants.

Non, c'est plutôt contre tout le public que je tiens à m'exprimer. Il a fait preuve à de nombreuses reprises, à mon avis, d'un manque certain de respect envers les intervenants. Alors même que le débat n'était censé avoir lieu qu'entre les intervenants, qui avaient certainement préparé leur plaidoyers et qui avaient été choisis de sorte à établir un certain équilibre entre les points de vue, on a pu entendre d'abord les sarcasmes intentionnellement non dissimulés, et ensuite les interventions inopinées d'un public venu dans l'unique optique de mobiliser tout le monde contre cette loi et non d'en comprendre les arguments en sa faveur. Car il y en a, m'a-t-il semblé.

C'est ainsi que les intervenants se sont vus confisquer la parole, ne s'autorisant pas à gueuler plus forts que certains membres du public ayant abandonné toute forme de respect envers eux et plus généralement toute personne ayant un avis différent.

Le public n'a pas écouté, ou du moins pas entendu la présence d'arguments intéressants, comme le fait que cette loi, à côté des éléments de surveillance qu'elle légalise, permet une réelle clarification de l'organisation des services de renseignements, et s'est efforcé de cantonner le débat aux boîtes noires et à d'autres points purement techniques qui, bien que catastrophiques pour les libertés individuelles, ne sont pas les seuls points intéressants du débat.

Ce débat était une occasion de découvrir toutes les facettes de cette loi, les raisons qui y ont poussé, les revendications du syndicat de la magistrature qui s'y oppose pour des raisons extrêmements différentes des raisons techniques entendues par le public, le retour concret et pragmatique de la façon d'agir de la DST et la génèse de cette loi. C'était une occasion de se défaire de la bulle idéologique dans laquelle gravite certainement une large partie du public et qui aurait dû chercher à découvrir autre chose que les points noirs ayant fait le tour des articles qu'ils lisent et qui proviennent de la même bulle.

Il a manqué une volonté de comprendre le point de vue des différents intervenants qui s'est soldée par un manque de respect honteux à l'égard de gens ayant passé du temps à réfléchir au problème qui est loin d'être aussi simple et binaire que certains le pensent.

Et enfin, je pense que ces débordements n'auraient pas pu avoir lieu si le public n'avait pas bénéficié de la complicité déplacée d'une modératrice complètement partiale et se moquant parfois presque des intervenants, n'hésitant pas à les couper sans vergogne et abusant du peu de pouvoir que son rôle lui conférait pour prendre des décisions autoritaire sur le ton de qui découvre qu'il a cette possibilité.

J'ai donc éprouvé une certaine honte vis à vis des intervenants. Honte d'appartenir à un public aussi peu diversifié et aussi peu enclin à s'intéresser à une quelconque diversité.

En espérant ne pas être le seul à avoir vécu ce débat d'une telle façon,
Élie Michel

A docker-like container management using systemd

10 min read

This article is an overview of how you can manage docker-like containers using systemd.

As you may know, the famous container manager docker is not the only container technology. It has been originally based on LXC, though it embed additional functionalities, but it mostly provides a high level API for container management. And actually it has been made possible by a bunch of quite recent kernel features such as namespaces, control groups or union filesystems.

As I was reading the excellent series of Lennart Poettering's articles about the use of systemd for administrators, I found out that systemd is actually able to handle its own containers.

It is not so surprising actually, since systemd is close to the kernel and so provides an abstraction over a lot of its features. But anyway, I did not know it was so easy-to-use!

I present in this article the basics of systemd-nspawn, the enhanced chroot that systemd brought to us to build containers.

Our first container

Let's start with some practical example, directly inspired from one of the Poettering articles. We are going to run a simple Debian inside the most basic container.

Set up files

We use debootstrap to pull a minimal Debian on our filesystem. This command is available on most linux distributions (well, I checked for apt, yum and yaourt, using AUR).

debootstrap --arch=amd64 unstable debian-tree/

It downloads Debian unstable files in the local debian-tree directory.

NB: I use Debian as an example, but using yum or pacstrap, you can install respectively fedora or arch as easily.

Run container

Just run systemd-nspawn (eventually as root):

systemd-nspawn --directory=debian-tree/

NB: You can use -D instead of --directory=

It performs some kind of chroot and open up a shell inside the hosted system. It not only isolates local filesystem from its host, but also abstracts its interfaces. It is much more powerfull than a simple chroot.

NB: By default /bin/sh is called inside the container, but you can specify the command you want systemd to run into the container as an extra argument. In that case, arguments specified after the command are forwarded to it and not treated by systemd-nspawn.

But we just mounted the Debian filesystem actually, and no more. If you call ps, you will not see any process but the shell (even if you are root).

root@debian-tree:~# ps
  PID TTY          TIME CMD
    1 ?        00:00:00 bash
    6 ?        00:00:00 ps

The Debian OS has not really started. Before we do so, think about setting a password for your root user, you will need it later:

root@debian-trash:~# passwd
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully

Booting container

Call systemd-nspawn with --boot option (or -b). Alternatively, you can ask systemd-nspawn to run /sbin/init:

systemd-nspawn -D debian-tree/ --boot
# or
systemd-nspawn -D debian-tree/ /sbin/init

You should see boot messages, with lines beginning with [ OK ]. Also, if you are hosting a distribution relying itself on systemd, e.g. a recent release of Debian, you would see interesting message at the early boot:

Spawning container debian-trash on /your/current/path/debian-trash.
Press ^] three times within 1s to kill container.
systemd 215 running in system mode. (+PAM +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ -SECCOMP -APPARMOR)
Detected virtualization 'systemd-nspawn'.
Detected architecture 'x86-64'.

[...]

You can notice that the hosted systemd detects that it has been ran inside a systemd-nspawn container! You can use systemd-detect-virt at any time to get this information.

Also note that you can escape the container (and so kill it) by pressing ^] (Control key plus ]) three times quickly, which can be useful, especially if you forgot to set any root password.

Managing containers

Docker provides a nice API to manage your containers. While systemd is not as exhaustive, it also have an interface for it, though the machinectl command (like "machine control").

$ machinectl
MACHINE                          CONTAINER SERVICE         
debian-tree                      container nspawn          

1 machines listed.

You can get more information about one particular container using machinectl status <machine>:

$ machinectl status debian-tree
debian-tree
           Since: mar. 2015-04-07 08:46:37 CEST; 55min ago
          Leader: 24553 (systemd)
         Service: nspawn; class container
            Root: /home/system/local/debian-tree
         Address: 192.168.1.108
                  fe80::2ae3:47ff:fe04:f134
              OS: Debian GNU/Linux 8 (jessie)
            Unit: machine-debian\x2dtree.scope
                  ├─24553 /lib/systemd/systemd
                  └─system.slice
                    ├─cron.service
                    │ └─24618 /usr/sbin/cron -f
                    ├─systemd-journald.service
                    │ └─24571 /lib/systemd/systemd-journald
                    ├─console-getty.service
                    │ ├─14910 man machinectl
                    │ ├─14921 pager -s
                    │ ├─24625 /bin/login --
                    │ └─24656 -bash
                    └─rsyslog.service
                      └─24620 /usr/sbin/rsyslogd -n

NB: If you want to process machinectl status output, please consider using machinectl show instead. It has been designed with this goal in mind.

If the container's content supports it, you can use machinectl reboot, machinectl poweroff or machinectl login for example. You can still use machinectl terminate to simply kill the whole container, whatever is running inside it.

About security

It is well known that LXC containers are still not secure enough to be used as absolute jails. Actually, this is due to possible exploits of kernel weakness that are present because the introduction of containers technologies is recent. If you need full isolation, consider using solution such as [KVM].

And as you can see it using systemd-cgtop (systemd's real time cgroup visualizer), some information about the host system leaks into the guest machine:

On the guest

Path                                     Tasks   %CPU   Memory  Input/s Output/s

/                                            -  144.6     2.3G       0B    94.5K
/machine.slice                               -    1.5    30.9M        -        -
/machine.sli...e-debian\x2dtree.scope       7    1.5    12.3M        -        -
/system.slice                                -      -    71.0M        -        -

On the host

Path                                     Tasks   %CPU   Memory  Input/s Output/s

/                                            -  144.6     2.3G       0B    94.5K
/machine.slice                               -    1.5    30.9M        -        -
/machine.sli...e-debian\x2dtree.scope        7    1.5    12.3M        -        -
/system.slice                                -      -    71.0M        -        -
/system.slice/NetworkManager.service         2      -        -        -        -
/system.slice/accounts-daemon.service        1      -        -        -        -
/system.slice/avahi-daemon.service           2      -        -        -        -
/system.slice/bluetooth.service              1      -        -        -        -
/system.slice/colord.service                 1      -        -        -        -
/system.slice/dbus.service                   1      -        -        -        -
/system.slice/gdm.service                    2      -        -        -        -
/system.slice/geoclue.service                1      -        -        -        -
/system.slice/httpd.service                  7      -        -        -        -
/system.slice/ipython-notebook.service       1      -        -        -        -
/system.slice/itorch-notebook.service        2      -        -        -        -
/system.slice/polkit.service                 1      -        -        -        -
/system.slice/postgresql.service             6      -        -        -        -
/system.slice/redis.service                  1      -        -        -        -
/system.slice/rtkit-daemon.service           1      -        -        -        -
/system.slic...lice/getty@tty2.service       1      -        -        -        -

For instance, you can access from the contained system to the resources used by the host! You can also see all other machine started, since they are by default on the same cgroup slice, namely /machine.slice.

And another important point is that by default the network interface is not virtualized: the container accesses the same interface than the host (try some ip link). But the good news is that we can easily manage to isolate it. And we can even add bridges between host and guest interfaces.

Network isolation

Complete isolation

If you do not need any network connection, you can completely isolate the container from its host's network using the --private-network option:

systemd-nspwan -bD debian-tree/ --private-network

If you look at the available network interfaces, you will see only a loopback:

guest$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

And actually, this loopback itself is isolated from the host's loopback interface, so there is absolutely no communication between host and guess (wrt. the previously advertised security limitations, of course).

Adding bridges

Anyway, it is common to need the container to be able to listen on some port. For instance, the container can host some web server, or socket served service. So we need to be able to expose some interface to the container.

Providing Internet interface

The simplest solution is to provide the container with a whole network interface.

systemd-nspwan -bD debian-tree/ --network-interface=eth0

Now you can see eth0 from within the container. But the main drawback is that this interface would not be accessible from the host: it is moved from one host namespace to guest namespace.

guest$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether **:**:**:**:**:** brd ff:ff:ff:ff:ff:ff

Note that the host loopback cannot be moved to the guest. Also not that you could add a second --network-interface argument to use many network interfaces.

NB: As soon as you specify that you would share a network interface, it automatically hides the other one, as if --private-network was specified.

You can then create bridged or vlan interfaces in the host and provide them to the container. Fortunately, systemd-spawn comes with some options simplifying this process.

Virtual Ethernet link

You can use --network-veth to create a virtual Ethernet link between host and container.

systemd-nspwan -bD debian-tree/ --network-veth

You will see a new interface on the host machine, named ve-debian-tree like "virtual ethernet to debian-tree container" and the container will be provided a host0 interface.

In order to communicate, you must then add an IP address to these new interfaces. For instance, you could use the reserved 10.0.0.1/8 IP block:

# Add address 10.0.0.1 to ve-debian-tree interface of host system
host$ ip addr add 10.0.0.1/24 broadcast 10.0.0.255 dev ve-debian-tree
host$ ip link set dev ve-debian-tree up
# Add address 10.0.0.2 to host0 interface of guest system
guest$ ip addr add 10.0.0.2/24 broadcast 10.0.0.255 dev host0
guest$ ip link set dev host0 up

You can then check you setup using ping 10.0.0.1 and ping 10.0.0.2 on guest and host respectively.

Now that you can communicate between host and guest, you may want to forward some host external ports toward guest system and block the other ones using for instance iptables.

Mount host directory within guest system

Although unlike docker containers, a systemd-nspawn container is persistent (modifications are not forget at reboot), it can be made non-persistent using --volatile=yes and anyway it can be useful to bind some host directory to some guest directory.

You can achieve this with --bind=/my/host/directory;/my/guest/directory.

# Boot container and mount host's /home into it
systemd-nspawn -bD debian-tree --bind=/home

Note that when both host and guest path are identical, you can omit the second one.

You can also mount it as read-only, using --bind-ro. Alternatively, you can use machinectl bind.

Dealing with images

I will end this overview by speaking about images, although reading [the man page][man:systemd-nspawn] of systemd-nspawn would show you a lot more settings.

I did not investigate a lot about images but there seems to be a complete image management available.

systemd-nspawn can take as argument a container images instead of e filesystem directory, using --image (or -i). These images can be for instance simple tarballs, but also docker images, and machinectl can help you pull images with commands such as machinectl pull-tar.

So I have to look further how I can use this power, but it seems that systemd provides us with a simple but powerful containing solution that can be deeply integrated with its other units.

References

Run Docker containers with systemd-nspawn | Sean McGary

Use as a container manager while still benefitting on existing images!

HackEns organise des conférences sur les libertés individuelles et Internet !

Flowhub - Peer-to-peer full-stack visual programming for your fingers | Flowhub

Mêler code et graphique : le site vitrine est superbe, mais je n'ai pas eu l'occasion de tester.

Notebook

Un exemples de navigation fluide dans un jeu de données grâce à Python.

Dev.Opera — In search of the perfect JavaScript framework

Un bon résumé des différentes caractéristiques à prendre en compte dans l'évaluation d'un Framework JavaScript, appuyé sur divers exemples.

Chris Granger - Coding is not the new literacy

Une excellente réflexion sur l'enseignement de l'informatique et ce qu'il est important de voir dans cette matière. L'enseignement de l'informatique ne doit pas se limiter à la programmation, qui n'est que l'expression de l'objectif sous-jacent qu'est la modélisation.

Globalement, l'article ne parle pas vraiment de trouver un aspect littéraire à la programmation, mais révèle des traits de la programmation pouvant être utiles à sa comparaison à la littérature. Pas de question de stylistique par exemple.

Finalement, l'article redit « Programming is the new literacy », mais en précisant ce que l'on doit tirer comme conclusion de cette phrase quant aux propriétés intéressantes de la programmation.

mrdoob/texgen.js · GitHub

Une bibliothèque très simple de génération procédurale d'images en JS.
Les exemples présentés sont très réussi, malgré la simplicité de la lib.