Emile "iMil" Heitor 's home

I’ve been obsessed with SaltStack for over a week. This infrastructure management suite is exactly what I needed for both my personal and professional servers: simple but modular, written in python, not depending on a thousand unnecessary complex messaging stacks as it bundles zeromq, capable of both orchestration and configuration management, all this through comprehensive, well documented API and commands.

Only drawback was it had poor NetBSD support. Was :)

It’s been a long time since I’ve dug into python, so it took me a little bit of effort, but Salt now has full support of pkgin in its generic packaging functions, knows how to handle NetBSD services and is capable of dealing with NetBSD‘s sysctl(8) and sysctl.conf.

Those pieces of code have been merged upstream, I hope they’ll be available in version 0.16!

Some examples:

$ cat packages/init.sls 
mypkgs:
  pkg.installed:
    - pkgs:
      - vim
      - tmux
      - bash
      - bash-completion
      - sudo

$ sudo salt '*' state.sls packages
watto:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:   
korriban:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:   
tatooine:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:   
coruscant:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:   
ragnos:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:   
exar:
----------
    State: - pkg
    Name:      mypkgs
    Function:  installed
        Result:    True
        Comment:   All specified packages are already installed.
        Changes:

$ sudo salt '*' cmd.run 'uname -a'
tatooine:
    Linux tatooine 3.2.0-4-686-pae #1 SMP Debian 3.2.41-2 i686 GNU/Linux
watto:
    NetBSD watto.home.imil.net 6.1_RC4 NetBSD 6.1_RC4 (GENERIC) i386
exar:
    NetBSD exar 6.0_STABLE NetBSD 6.0_STABLE (EXAR) #0: Sun Nov 25 12:39:12 CET 2012  root@exar:/usr/src/sys/arch/i386/compile/EXAR i386
coruscant:
    NetBSD coruscant 6.0 NetBSD 6.0 (XEN3_DOM0) amd64
korriban:
    NetBSD korriban.imil.net 6.0_STABLE NetBSD 6.0_STABLE (KORRIBAN) #0: Tue Jan  1 23:20:36 CET 2013  root@korriban.imil.net:/usr/src/sys/arch/amd64/compile/KORRIBAN amd64
ragnos:
    NetBSD ragnos 6.0 NetBSD 6.0 (RAGNOS) #2: Wed Oct 17 11:33:31 CEST 2012  root@ragnos:/usr/src/sys/arch/i386/compile/RAGNOS i386

$ sudo salt '*' pkg.version vim   
watto:
    7.3.762
exar:
    7.3.762
korriban:
    7.3.712
coruscant:
    7.3.762
ragnos:
    7.3.762
tatooine:
    2:7.3.547-7

$ sudo salt '*' service.status sshd
tatooine:
    False
watto:
    True
coruscant:
    True
exar:
    True
korriban:
    True
ragnos:
    True

If you whish to use these modules without tainting your Salt package installation, simply copy them to a _modules directory within the file_roots.

Happy Salting!

I know about estd for a while, that daemon “dynamically sets the CPU-frequency on Enhanced SpeedStep, PowerNow, and APCI P-States-enabled CPUs depending on current cpu-utilization” (manpage excerpt). Thing is, I’ve never seen any CPU changing from its current speed while monitoring the machdep.powernow.frequency.current sysctl.

In order to understand what was happening, I started estd with the -o flag, which outputs the CPU-frequencies as they are set. I then realized that the “ligh watermark percentage” and “low watermark percentage” default values were way too high (respectively 40 and 80) and were never reached, so the CPU speed was never changed.

With lower values, I was able to see the CPU speed increasing and lowering as expected. So I added the following line to the /etc/rc.conf file:

estd_flags="-l 5 -h 15 -a -m 800 -d"

meaning that the low watermark is set at 5 and the high watermark at 15, which were the values I’ve considered being the right ones while watching estd -o -a console output.

Since then, whenever a CPU intensive operation occurs, I can see the CPU speed rising with the following conky parameter:

CPU Frequency: ${alignr}${exec /sbin/sysctl -n machdep.powernow.frequency.current}

Update

As written on the comments:

Colin Watson Says:
May 17th, 2013 at 7:12 pm

I uploaded 6.2 packages to Debian a week or so after you posted this, so you can/should now just use those instead. I expect they should build fine on wheezy.

As a matter of fact, the following is now deprecated

At ${DAYWORK}, we used to have our own OpenSSH debian package which included the famous OpenSSH LPK patch, which permits the use of an OpenLDAP server as an SSH public key provider.

I’ve been using OpenSSH-LPK for years, as this is a really handy solution and no valid alternative existed… until a couple of months.

OpenSSH 6.2 has a new configuration item called “AuthorizedKeysCommand”. The value associated to that key permits to call any executable as a public key provider. Yes, that is sexy.

Debian only have OpenSSH 6.1p1 packages available and tagged as “experimental”, so we had to hack a little bit in order to build 6.2 packages, here’s how:

• Fetch experimental source package

# echo "deb-src http://ftp2.fr.debian.org/debian/ experimental main contrib non-free" > /etc/apt/sources.list.d/experimental.list
# apt-get update
$ mkdir openssh && cd openssh
$ apt-get source openssh

• Bump the release

$ wget http://ftp.fr.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-6.2p1.tar.gz
$ cd openssh-6.1p1
$ uupdate -v 6.2p2 ../openssh-6.2p1.tar.gz
$ cd ../openssh-6.2p1
$ dch -i # enter changelog informations

• Get rid of conflicting patches

As expected, many patches from debian don’t apply anymore, and I was not brave enough to backport them, I’ve just commented them in debian/patches/series:

#gssapi.patch
#selinux-role.patch
#copy-id-restorecon.patch
#ssh-vulnkey.patch
#consolekit.patch
#user-group-modes.patch
#max-startups-default.patch
#package-versioning.patch
#debian-banner.patch
#lintian-symlink-pickiness.patch
#openbsd-docs.patch
#ssh-argv0.patch
#doc-upstart.patch

• Remove uninstalled files

In order not to check some files that will not be present as we commented the patches which creates them, we’ll have to remove the following lines from debian/openssh-client.install

usr/bin/ssh-vulnkey
usr/share/man/man1/ssh-vulnkey.1

And the following one from debian/openssh-client.docs

ChangeLog.gssapi

Finally, we just comment out the use of the vulnerable_host_keys shell function in debian/openssh-server.postinst.in:

fix_doc_symlink
create_sshdconfig
create_keys
#vulnerable_host_keys
fix_statoverride

That’s it! You can now happily build the brand new OpenSSH version using debuild as usual.

After installing it, you’ll have access to the AuthorizedKeysCommand option.

Thanks gaston, davromaniak and SliX from #GCU for the help.

Here’s a .vimrc excerpt which permits the use of Control + Arrows sequences to navigate through vim tabs inside the tmux terminal multiplexer:

" inside screen / tmux
map <Esc>[C <C-Right>
map <Esc>[D <C-Left>
" insert mode
map! <Esc>[C <C-Right>
map! <Esc>[D <C-Left>
" no screen
map <Esc>[1;5D <C-Left>
map <Esc>[1;5C <C-Right>
" insert mode
map! <Esc>[1;5D <C-Left>
map! <Esc>[1;5C <C-Right>

nnoremap <C-t> :tabnew<CR>
nnoremap <C-w> :tabclose<CR>
nnoremap <C-right> :tabnext<CR>
nnoremap <C-left> :tabprevious<CR>
" insert mode
inoremap <C-t> <Esc>:tabnew<CR>
inoremap <C-w> <Esc>:tabclose<CR>
inoremap <C-right> <Esc>:tabnext<CR>
inoremap <C-left> <Esc>:tabprevious<CR>

I recently switched pkgin‘s repository from SourceForge‘s CVS to GitHub. Long story short, I heard here and there that SF was considering to drop CVS support and I found GitHub service to be more responsive and elegant. Also, I was looking for an excuse to learn git :)

Anyway, GitHub interface may be sexy, they used to have some kind of “upload” section which has been dropped. That may sound like a simple story, but the fact is when it comes to packaging a GitHub-hosted application, things are not that simple when the author has not explicitly tagged a specific release. Another use case, in which I actually am, is when you have an ongoing development, like pkgin in pkgsrc WIP and do not want to tag every test-release.

The way I found to handle that case with pkgsrc is to use GitHub’s commit archives. In short, I will use that kind of URL:

https://github.com/NetBSDfr/pkgin/archive/34b823c158e62e4d347de74499a075a2259382c5.tar.gz

which is redirected like this by GitHub:

HTTP/1.1 302 Found
Server: GitHub.com
Date: Sun, 21 Apr 2013 21:05:17 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Status: 302 Found
Cache-Control: max-age=0, private
Strict-Transport-Security: max-age=2592000
X-Frame-Options: deny
Set-Cookie: logged_in=no; domain=.github.com; path=/; expires=Thu, 21-Apr-2033 21:05:17 GMT; HttpOnly
Location: https://nodeload.github.com/NetBSDfr/pkgin/tar.gz/34b823c158e62e4d347de74499a075a2259382c5
X-Runtime: 13
Content-Length: 156
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: GitHub.com
Date: Sun, 21 Apr 2013 21:05:18 GMT
Content-Type: application/x-gzip
Connection: keep-alive
Content-Length: 187510
Content-Disposition: attachment; filename=pkgin-34b823c158e62e4d347de74499a075a2259382c5.tar.gz
Vary: Accept-Encoding

and permits to point to a particular commit, no matter if it has been tagged or not.

A typical pkgsrc Makefile will look like this:

VERSION=                34b823c158e62e4d347de74499a075a2259382c5
DISTNAME=               ${VERSION}
PKGNAME=                pkgin-20130412
CATEGORIES=             pkgtools
MASTER_SITES=           https://github.com/NetBSDfr/pkgin/archive/
FETCH_USING=            curl

# [...]

WRKSRC=                 ${WRKDIR}/pkgin-${VERSION}

# [...]

Note that FETCH_USING= curl is mandatory here in order to follow redirect codes along with https.

There you go, happy GitHub packaging!

Update

Here’s another approach pointed out by Amitai Schlair (schmonz@):

GIT_COMMIT=	dd51ac5

DISTNAME=	${GIT_COMMIT}
PKGNAME=	p5-App-Prove-Plugin-ProgressBar-0.01
CATEGORIES=	devel perl5
MASTER_SITES=	-http://nodeload.github.com/Ovid/App-Prove-Plugin-ProgressBar/tar.gz/${GIT_COMMIT}

# [...]

WRKSRC=		${WRKDIR}/App-Prove-Plugin-ProgressBar-${GIT_COMMIT}

Here, Amitai doesn’t use HTTPS so specifying curl as the fetch method is not mandatory. The dash before the URL in the MASTER_SITES line means that DISTNAME will not be appended when fetching, which is very handy when it comes to GitHub archives.

Il est là, il est chaud, et il contient deux articles de mon cru:

• 3NMP: NetBSD, Nginx, Naxsi, MySQL, PHP
• Nouvelles commandes et nouveaux démons dans NetBSD 6.0

Enjoy!