Emile “iMil” Heitor ‘s home

!@#!@# de bug.

Des mois que je cherchais au mauvais endroit, en effet, uniquement chez certains utilisateurs (évidemment), un pkgin up / fug proposait d’installer l’univers. Problème dans le moteur de dépendances me dis-je; que de temps perdu à décortiquer ce qui fonctionnait parfaitement, pour finalement tomber sur ceci:

static const struct Summary {
	const int	type;
	const char	*tbl_name;
	const char	*deps;
	const char	*conflicts;
	const char	*requires;
	const char	*provides;
	const char	*end;
} sumsw[] = {
	{
		LOCAL_SUMMARY,
		"LOCAL_PKG",
		"LOCAL_DEPS",
		"LOCAL_CONFLICTS",
		"LOCAL_REQUIRES",
		"LOCAL_PROVIDES",
		NULL
	},
	{
		REMOTE_SUMMARY,
		"REMOTE_PKG",
		"REMOTE_DEPS",
		"REMOTE_CONFLICTS",
		"REMOTE_REQUIRES",
		"REMOTE_PROVIDES",
		NULL
	},
};

[...]

	for (ptbl = __UNCONST(sum.tbl_name), i = 0;
		 i < nelms;
		 ptbl += ((strlen(ptbl) + 1) * sizeof(char)), i++) {

Il ne faut PAaaaaaaas coder avec de l'alcool dans le sang, il ne faut pas, parce qu'après, on se retrouve avec des bouts de trucs ni faits ni à faire qui supposent que les membres d'une structure seront forcément alignés, et des effets de bords qui n'ont rien à voir.

Bref, stacktic m'a proposé cette méthode, bien plus élégante:

static const struct Summary {
	const int	type;
	const char	*tbl_name;
	const char	*deps;
	const char	*conflicts;
	const char	*requires;
	const char	*provides;
	const char	*end;
} sumsw[] = {
	[LOCAL_SUMMARY] = {
		LOCAL_SUMMARY,
		"LOCAL_PKG",
		"LOCAL_DEPS",
		"LOCAL_CONFLICTS",
		"LOCAL_REQUIRES",
		"LOCAL_PROVIDES",
		NULL
	},
	[REMOTE_SUMMARY] = {
		REMOTE_SUMMARY,
		"REMOTE_PKG",
		"REMOTE_DEPS",
		"REMOTE_CONFLICTS",
		"REMOTE_REQUIRES",
		"REMOTE_PROVIDES",
		NULL
	},
};

[...]

	/* (REMOTE[LOCAL)_PKG is first -> skip */
	for (arr = &(sum.tbl_name) + 1; *arr != NULL; ++arr) {

Aaah, bah oui, tout de suite c’est moins dégueulasse hein.

0.5.2.3 est dans current, et le pullup request vient de partir.

meh.

I’ve just commited 0.5.2.1. As the version shows, it is a bugfix release; is@ got a very nasty bug that took me a while to figure out. For 3 years, the only pkg_summary(5) format I’ve seen was:

PKGNAME=foo
[...]

or, in case of conflicting packages:

CONFLICTS=bar
CONFLICTS=baz
PKGNAME=foo
[...]

Well it turns out that we can also find:

PKGNAME=foo
CONFLICTS=bar
[...]

which is pretty annoying when it comes to stick to an anchor. Until now, I did the following:

static int
chk_pkgname(char *field)
{
        if (strncmp(field, "PKGNAME=", 8) == 0 ||
                strncmp(field, "CONFLICTS=", 10) == 0)
                return 1;

        return 0;
}
[...]
                /* browse entries following PKGNAME and build the SQL query */
                while (*psum != NULL && !chk_pkgname(*psum)) {
                        update_col(sum, pkgid, *psum);
                        psum++;
                }

But with that new case, I had to check a little bit further:

static int
chk_pkgname(char *field, char *last_field)
{
	if (strncmp(field, "PKGNAME=", 8) == 0)
		return 1;
	/* in some very rare cases, CONFLICTS appears *after* PKGNAME */
	if (strncmp(last_field, "PKGNAME=", 8) != 0 &&
		/* never seen many CONFLICTS after PKGNAME, but just in case... */
		strncmp(last_field, "CONFLICTS=", 10) != 0 &&
		strncmp(field, "CONFLICTS=", 10) == 0)
		return 1;

	return 0;
}
[...]
		/* browse entries following PKGNAME and build the SQL query */
		while (*psum != NULL && !chk_pkgname(*psum, *(psum - 1))) {
			update_col(sum, pkgid, *psum);
			psum++;
		}

I really don’t like the way it’s done but I’ve no sexier way in mind at the moment. Maybe after some glasses of wine…

Finally, here comes a release ! Read the full announcement and changelog on pkgsrc-users.

And here we are for week 2 !

There have been some fixes last week, issues were mainly spotted by orgrim, thanks to him !

The big one concerns a type of dewey/glob I’ve never been through before:

libao-[a-z]*-[0-9]*

Until now, I was able to handle the following (from pkg_str.c)

/*
 * AFAIK, here are the dewey/glob we can find as dependencies
 *
 * foo>=1.0 - 19129 entries
 * foo<1.0 - 1740 entries (only perl)
 * foo>1.0 - 44 entries
 * foo< =2.0 - 1
 * {foo>=1.0,bar>=2.0}
 * foo>=1.0<2.0
 * foo{-bar,-baz}>=1.0
 * foo{-bar,-baz}-[0-9]*
 * foo-{bar,baz}
 * foo-1.0{,nb[0-9]*} - 260
 * foo-[0-9]* - 3214
 * foo-1.0 - 20
 */

As there was no “easy” way of handling those double-globs dependencies, I took a simple approach: resolve dependency with a matching package when the pattern is too complex (sqlite_callback.c):

    /* map corresponding pkgname */
    if ((pkg_map = map_pkg_to_dep(plisthead, deptree->depend)) != NULL)
        XSTRDUP(deptree->name, pkg_map->name);
    else
        /* some dependencies just don't match anything */
        XSTRDUP(deptree->name, DEPS_PKGNAME);

Only drawback, this takes a bit more time, but at last, there’s no possible error on dependency resolution. Maybe there’s some more optimizations I can do to reduce time taken with this mapping.

As planned, pkgin now depends on pkgsrc’s pkg_install, and the latter is pushed on top of the upgrade-list if there’s a new version.
I also fixed the many-versions-of-the-same-package problem with the following query (pkgindb_queries.c)

const char DIRECT_DEPS[] = /* prefer higher version */
    "SELECT REMOTE_DEPS_DEWEY, REMOTE_DEPS_PKGNAME "
    "FROM REMOTE_DEPS WHERE PKG_ID = "
    "(SELECT PKG_ID FROM REMOTE_PKG WHERE PKGNAME = '%s' "
    "ORDER BY FULLPKGNAME DESC LIMIT 1);";

Needs to be done:

• Tests, tests, tests !
• reproduce and fix 2 different bugs two users had; I’ve been unable to reproduce them at the moment
• test Minix 3.2.0

Foreword: this post will be written in english as many pkgin users don’t speak french. Sorry to my french readers then, and sorry also to my english readers as i’m not as fluent in english as i am in french :)

I subscribed to jmmv’s blog, The Julipedia, a while ago and found his idea of the “Kyua: Weekly status report” very inspiring, that’s a good way to keep your users informed on how the project is moving and keeps you focused on your TODO (although i hate TODO’s…). I doubt i’ll have the time to write a weekly report, but at last i’ll try to write a post whenever important updates are made to my beloved project.

So let’s begin !
As you, pkgin users, may be aware of, i’m working on the future 0.5 release, which includes massive internal changes plus some features that have been asked and i found interesting. So, in addition to the features i already listed in this mail sent to pkgsrc-users@, here are some hilights on recent changes:

• pkgin now has a new logo !
• it is now possible to export / import your keep-list, pretty much like dpkg‘s get/set-selection. The exported list is in pkg_chk‘s format. Thanks wiz@ for the idea
• pkgin install can now take a “blob” as an argument, i.e. pkgin in 'mysql-server<5.5', thanks filip@ for the idea
• pkgin now uses pkgsrc’s pkg_install for NetBSD also
• added the -t modifier, mostly for debugging purposes, in order to trace the dependency tree and impact lists
• pkg_install error logs are handled in a nicer way
• enlisted pkgin’s code to ohloh (click on “i use this” !)
• plus usual bugfixes

Yeah, these were fairly productive holidays :) Of course most of these changes are only available in CVS, see http://pkgin.net for details. Some of them have already made their way to pkgsrc-wip, i try not to insert big changes now, so wip and CVS should be sync’ed quite often.

Needs to be done:

• make pkgin’s pkgsrc package depend on pkgsrc’s pkg_install
• reproduce and fix 2 different bugs two users had
• optimize the dependency loop regarding packages that exists in many versions (i.e. bash)
• check if pkg_install is to be upgraded and then push it on top of ordered list
• test Minix 3.2.0

Hope i’ll make it to pkgsrc 2011Q4 !

J’aime aussi les vacances parce qu’elles me donnent des idées, et ce matin, alors que j’attendais que la piscine se réchauffe, j’ai pondu ça: