Emile "iMil" Heitor 's home

I’ve been obsessed with SaltStack for over a week. This infrastructure management suite is exactly what I needed for both my personal and professional servers: simple but modular, written in python, not depending on a thousand unnecessary complex messaging stacks as it bundles zeromq, capable of both orchestration and configuration management, all this through comprehensive, well documented API and commands.

Only drawback was it had poor NetBSD support. Was :)

It’s been a long time since I’ve dug into python, so it took me a little bit of effort, but Salt now has full support of pkgin in its generic packaging functions, knows how to handle NetBSD services and is capable of dealing with NetBSD‘s sysctl(8) and sysctl.conf.

Those pieces of code have been merged upstream, I hope they’ll be available in version 0.16!

Some examples:

$ cat packages/init.sls 
mypkgs:
  pkg.installed:
    - pkgs:
      - vim
      - tmux
      - bash
      - bash-completion
      - sudo

$ sudo salt '*' state.sls packages
watto:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:   
korriban:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:   
tatooine:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:   
coruscant:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:   
ragnos:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:   
exar:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:

$ sudo salt '*' cmd.run 'uname -a'
tatooine:
    Linux tatooine 3.2.0-4-686-pae #1 SMP Debian 3.2.41-2 i686 GNU/Linux
watto:
    NetBSD watto.home.imil.net 6.1_RC4 NetBSD 6.1_RC4 (GENERIC) i386
exar:
    NetBSD exar 6.0_STABLE NetBSD 6.0_STABLE (EXAR) #0: Sun Nov 25 12:39:12 CET 2012  root@exar:/usr/src/sys/arch/i386/compile/EXAR i386
coruscant:
    NetBSD coruscant 6.0 NetBSD 6.0 (XEN3_DOM0) amd64
korriban:
    NetBSD korriban.imil.net 6.0_STABLE NetBSD 6.0_STABLE (KORRIBAN) #0: Tue Jan  1 23:20:36 CET 2013  root@korriban.imil.net:/usr/src/sys/arch/amd64/compile/KORRIBAN amd64
ragnos:
    NetBSD ragnos 6.0 NetBSD 6.0 (RAGNOS) #2: Wed Oct 17 11:33:31 CEST 2012  root@ragnos:/usr/src/sys/arch/i386/compile/RAGNOS i386

$ sudo salt '*' pkg.version vim   
watto:
    7.3.762
exar:
    7.3.762
korriban:
    7.3.712
coruscant:
    7.3.762
ragnos:
    7.3.762
tatooine:
    2:7.3.547-7

$ sudo salt '*' service.status sshd
tatooine:
    False
watto:
    True
coruscant:
    True
exar:
    True
korriban:
    True
ragnos:
    True

If you whish to use these modules without tainting your Salt package installation, simply copy them to a _modules directory within the file_roots.

Happy Salting!

I know about estd for a while, that daemon “dynamically sets the CPU-frequency on Enhanced SpeedStep, PowerNow, and APCI P-States-enabled CPUs depending on current cpu-utilization” (manpage excerpt). Thing is, I’ve never seen any CPU changing from its current speed while monitoring the machdep.powernow.frequency.current sysctl.

In order to understand what was happening, I started estd with the -o flag, which outputs the CPU-frequencies as they are set. I then realized that the “ligh watermark percentage” and “low watermark percentage” default values were way too high (respectively 40 and 80) and were never reached, so the CPU speed was never changed.

With lower values, I was able to see the CPU speed increasing and lowering as expected. So I added the following line to the /etc/rc.conf file:

estd_flags="-l 5 -h 15 -a -m 800 -d"

meaning that the low watermark is set at 5 and the high watermark at 15, which were the values I’ve considered being the right ones while watching estd -o -a console output.

Since then, whenever a CPU intensive operation occurs, I can see the CPU speed rising with the following conky parameter:

CPU Frequency: ${alignr}${exec /sbin/sysctl -n machdep.powernow.frequency.current}

And voila, I knew this was going to happen, I finally switched to tmux. I will not explain here why this software is better than GNU screen, simply put, tmux is now part of the base system of nearly all BSD derivative operating systems (NetBSD among them), and it makes my life easier.

Instead, I’ll point out here all the resources that helped me switching rather quickly:

• Generic hands-on https://wiki.archlinux.org/index.php/Tmux
• Commented example http://www.doknowevil.net/2010/10/18/sorry-screen-tmux-is-better-but-here-are-some-screen-lik-hotkeys/
• Playing with the statusbar http://jasonwryan.com/blog/2011/06/10/setting-tmux-statusbar-if-in-x/, this author has other very good resources on the topic
• Fixing the 256colors mess (without using tmux -2) http://benjaminthomas.org/2012-03-28/256-colors-in-tmux.html
• Another well documented configuration file http://www.sheevaboite.fr/articles/show-me-your-tmux-conf
• Screen-to-tmux helper (in french, but easy to comprehend) http://wiki.soolbox.net/wiki/SysadminTips/tmux
• Very well written customization guide http://blog.hawkhost.com/2010/07/02/tmux-%E2%80%93-the-terminal-multiplexer-part-2/ (author has also written a comprehensive guide on how to basically use tmux)
• Enable mouse pane focus and resize http://tangledhelix.com/blog/2012/07/16/tmux-and-mouse-mode/

Here’s my ~/.tmux.conf, which is pretty much the standard screen-keys.conf available with the package, with only some small customizations like the status bar and some fixes:

# Set the prefix to ^Q.
unbind C-b
set -g prefix ^Q
bind q send-prefix

# Bind appropriate commands similar to screen.
# lockscreen ^X x 
unbind ^X
bind ^X lock-server
unbind x
bind x lock-server

# screen ^C c 
unbind ^C
bind ^C new-window
unbind c
bind c new-window

# detach ^D d
unbind ^D
bind ^D detach

# displays * 
unbind *
bind * list-clients

# next ^@ ^N sp n 
unbind ^@
bind ^@ next-window
unbind ^N
bind ^N next-window
unbind " "
bind " " next-window
unbind n
bind n next-window

# title A
unbind A
bind A command-prompt "rename-window %%"

# other ^A
unbind ^A
bind ^A last-window

# prev ^H ^P p ^? 
unbind ^H
bind ^H previous-window
unbind ^P
bind ^P previous-window
unbind p
bind p previous-window
unbind BSpace
bind BSpace previous-window

# windows ^W w 
unbind ^W
bind ^W list-windows
unbind w
bind w list-windows

# quit \ 
unbind \
bind \ confirm-before "kill-server"

# kill K k 
unbind K
bind K confirm-before "kill-window"
unbind k
bind k confirm-before "kill-window"

# redisplay ^L l 
unbind ^L
bind ^L refresh-client
unbind l
bind l refresh-client


# :kB: focus up
unbind Tab
bind Tab select-pane -t:.+
unbind BTab
bind BTab select-pane -t:.-

# split
unbind |
bind | split-window -h
unbind -
bind - split-window -v

# " windowlist -b
unbind '"'
bind '"' choose-window

unbind r
bind r source ~/.tmux.conf

set -g terminal-overrides 'xterm*:smcup@:rmcup@:colors=256'
# Status Bar
set -g default-terminal "screen"

# default statusbar colors
set -g status-fg white
set -g status-bg colour235

# current or active window in status bar
set-window-option -g window-status-current-bg colour53
set-window-option -g window-status-current-fg white
set-window-option -g window-status-current-format '[ #W ]'

# alerted window in status bar (bell, activity or content).
set-window-option -g window-status-alert-bg colour235
set-window-option -g window-status-alert-fg colour53
set-window-option -g window-status-alert-attr reverse

set -g status-right-length 40
set -g status-left-length 40
#set -g status-left '#[fg=purple]#H #[fg=black,bright]#[default]'
set -g status-left '#[fg=colour153]#H #[default]'
set -g status-right '#[fg=colour141]#(sysctl vm.loadavg|cut -f2-4 -d" ")#[default] | #[fg=colour117]%H:%M '

In order to pick up the right colors, I used a shell snippet I took from this github repository:

for i in {0..255} ; do
    printf "\x1b[38;5;${i}mcolour${i}\n"
done

Mandatory screenshot:

I’ve been searching for a Web-based SSH for quite a while, and I recently read about GateOne on the dedicated Wikipedia page. Not only GateOne does what I was searching for, but it also comes up with nice features like interpreting images on-the-fly.

The remote server on which I intended to run this software is, of course, a NetBSD 6.0 domU, and as I anticipated, some work was needed in order to make GateOne work on this platform.

The software needs the following dependencies:

• textproc/py-html5lib
• devel/py-readline
• graphics/py-imaging
• misc/dtach
• www/py-tornado

The latter was not available in pkgsrc, so I created the package and commited it to pkgsrc-current. The package is straightforward:

# $NetBSD: Makefile,v 1.1 2013/01/25 22:33:47 imil Exp $
#

DISTNAME=       tornado-2.4.1
PKGNAME=        ${PYPKGPREFIX}-${DISTNAME}
CATEGORIES=     www
MASTER_SITES=   https://github.com/downloads/facebook/tornado/

MAINTAINER=     imil@NetBSD.org
HOMEPAGE=       http://www.tornadoweb.org/
COMMENT=        Fast and non-blocking web framework
LICENSE=        apache-2.0

FETCH_USING=    curl

USE_PKGLOCALEDIR=       yes

REPLACE_PYTHON+=        setup.py
REPLACE_PYTHON+=        tornado/*.py
REPLACE_PYTHON+=        tornado/platform/*.py
REPLACE_PYTHON+=        tornado/test/*.py

.include "../../lang/python/egg.mk"
.include "../../lang/python/application.mk"
.include "../../mk/bsd.pkg.mk"

Back to GateOne, while this software is really handy, it has not been made very “packageable”, meaning that it expects everything to be located on the same directory, i.e. ${PREFIX}/gateone. Not being brave enough to dig into the implications of such choice within the code, I have not made a public package of it, instead I created a local-package that suits my needs. Why not simply use the bundled setup.py? Because every .py file begins with:

#!/usr/bin/env python

this had to be changed to pkgsrc’s default python PATH, and using pkgsrc’s REPLACE_PYTHON command was easier than a find / sed / whatever oneliner.

Here’s the local package I created in pkgsrc/localpkg, which is owned by my user for convenience:

# $NetBSD$
#

DISTNAME=   gateone-1.1
CATEGORIES= shells
MASTER_SITES=   https://github.com/downloads/liftoff/GateOne/

MAINTAINER= imil@NetBSD.org
HOMEPAGE=   https://github.com/liftoff/GateOne/archive/
COMMENT=    Web Terminal Emulator and SSH Client
#LICENSE=   TODO

FETCH_USING=    curl

WRKSRC=         ${WRKDIR}/GateOne
USE_PKGLOCALEDIR=   yes
USE_LANGUAGES=      # none

REPLACE_PYTHON+=    *.py */*.py */*/*.py */*/*/*.py

# change this to the base directory you want gateone to land on !
BASEDIR=            /home/imil/www

SUBST_CLASSES+=     path
SUBST_STAGE.path=   pre-configure
SUBST_FILES.path+=      setup.py
SUBST_SED.path=         -e "s,/opt,${PREFIX},g"

PYSETUPINSTALLARGS= --prefix=${PREFIX}

DEPENDS+=   ${PYPKGPREFIX}-html5lib>=0.90:../../textproc/py-html5lib
DEPENDS+=   ${PYPKGPREFIX}-tornado>=2.4.1:../../www/py-tornado
DEPENDS+=   dtach>=0.8:../../misc/dtach

.include "../../devel/py-readline/buildlink3.mk"
.include "../../graphics/py-imaging/buildlink3.mk"
.include "../../lang/python/distutils.mk"
.include "../../lang/python/application.mk"
.include "../../mk/bsd.pkg.mk"

At some point I may write a real package out of this, but I wanted to play with that shiny client quickly :)

Once installed, you’ll end up with a ${BASEDIR}/gateone directory, which you may want to chown -R youruser for convenience.

I rapidly got bitten by that reported bug https://github.com/liftoff/GateOne/issues/161, which actually was a bug in the software itself. Simply modify the file /usr/pkg/gateone/plugins/ssh/scripts/ssh_connect.py, line 402, from:

os.execvpe(script_path, [], env)

to

os.execvpe(script_path, [''], env)

At this point, GateOne should run normally. If you are, as I do, to run the frontend with a non-privileged user, take care to change some values in the server.conf file:

port = 1443
pid_file = "./gateone.pid"
unix_socket_path = "./gateone.sock"

Also, as I use a CAcert-enabled reverse proxy, I had no use of the SSL, so I set disable_ssl to True. Do not forget to update the origins variable accordingly.

There we go! I now have a shiny HTML5 Terminal Emulator, ready to accept my SSH connections wherever I am… and here’s the mandatory screenshot :)

And voila! iMil.net has now migrated to a brand new (well, actually recycled) server, which is incidentally hosted by myself, in my company’s server room.

What are the news? on the architecture side, nothing revolutionary, my good old setup composed of a Debian (squeeze, yeah I don’t like to play) GNU/Linux dom0, which hosts various NetBSD 6.0/amd64 domUs (now SMP!).

Main news is the activation of naxsi, the Web Application Firewall on the nginx reverse proxy. I don’t like to waste IPv4 public addresses, so the websites I host are all served by an nginx reverse proxy that connects to domUs private IPs. Naxsi‘s rules are detailed in this post. Apart from that, nginx configuration is rather classic, here’s a vhost example:

server {
        server_name foo.com www.foo.com;
        listen [::]:80;

        access_log      /var/log/nginx/www.foo.com-access.log;
        error_log       /var/log/nginx/www.foo.com-error.log;

        include denied;

        location / {
                include /etc/nginx/naxsi.rules;

                proxy_pass      http://192.168.1.1:80;
                proxy_redirect  off;
                proxy_pass_header       Set-Cookie;
                # no cache for this one
        }
}

denied is used by naxsi, it’s a very simple location:

$ cat denied 
location /denied {
        root /var/www;
}

As I said before, NetBSD domUs are running 6.0.1. The main (mine) domU starts the following main services :

• Apache 2.4 + PHP 5.4
• Sendmail 8.14.5 (please shut your mouth on this one unless you know what you’re talking about)
• Dovecot 2.1.12
• MySQL Server 5.1.65
• Asterisk 1.8
• OpenVPN 2.2.2
• Bind 9.9.1-P4

Needless to say everything, except bind which is provided in base, was installed in binary form using… pkgin ;) The packages came from pkgsrc-2012Q3 available at packages.netbsdfr.org.

Each service needing a public facing interface is made accessible with iptables -t nat -A PREROUTING from the dom0.

As it already was the case, the website is available with both IPv4 and IPv6 IPs, maybe I’ll include some easter eggs for the latter :)

There you go. I hope you’ll continue enjoying my (now displaying faster) posts :)

GNU/Linux Magazine France numéro 156 est sorti, vous y trouverez un article de mon cru sur l’histoire et l’utilisation de l’infrastructure rc.d. Bonne lecture !