libexec?

[David Faure]

Does Qt have the concept of a libexec directory?
I.e. a directory that contains executables that are used by library code, and
that shouldn't be in $PATH so as not to bother users.

I'm porting code to QStandardPaths::findExecutable, and realizing that it only
looks in PATH, so it doesn't have support for other directories like libexec.
But I don't see how it could "guess" where to look, since this isn't typically
determined by an environment variable...

(For an example of what this is all about, ls /usr/lib*/kde4/libexec/)

[Thiago Macieira]

Post by David Faure
Does Qt have the concept of a libexec directory?
I.e. a directory that contains executables that are used by library code,
and that shouldn't be in $PATH so as not to bother users.
I'm porting code to QStandardPaths::findExecutable, and realizing that it
only looks in PATH, so it doesn't have support for other directories like
libexec. But I don't see how it could "guess" where to look, since this
isn't typically determined by an environment variable...
(For an example of what this is all about, ls /usr/lib*/kde4/libexec/)

The KDE 4 libexec is also new in KDE 4. In KDE 3 we used to pollute /usr/bin
with out helper tools. I still want to run kdontchangethehostname every now
and then to see what it does :-)

I think the concept is a valid one and should be reflected in QStandardPaths. I
just don't know where to get the list of paths.

As it stands right now, Qt has no helper executables, so we've never needed to
find them from Qt code. Applications and other libraries have done that for
themselves.

Tell me this: how would you write code that searches for a generic libexec (no
"kde4" in the pathname) and yet allows KDE to find its helper tools?

[Sune Vuorela]

Post by Thiago Macieira
Tell me this: how would you write code that searches for a generic libexec (no
"kde4" in the pathname) and yet allows KDE to find its helper tools?

And even write one generic enough to work across distributions.

Debian and friends uses more or less /usr/lib/NAME/libexec
where as fedora and friends uses /usr/libexec/NAME/

/Sune

[Thiago Macieira]

Post by Sune Vuorela

Post by Thiago Macieira
Tell me this: how would you write code that searches for a generic libexec
(no "kde4" in the pathname) and yet allows KDE to find its helper tools?

And even write one generic enough to work across distributions.
Debian and friends uses more or less /usr/lib/NAME/libexec
where as fedora and friends uses /usr/libexec/NAME/

The big difference there is whether those executables are architecture-specific
in multiarch configurations (e.g., /usr/lib64/NAME/libexec). Usually they
aren't.

/usr/libexec is unusual for us in the Qt world, but it's allowed by the FHS.
Unlike /usr/imports...

[David Faure]

Post by Thiago Macieira

Post by Sune Vuorela

Post by Thiago Macieira
Tell me this: how would you write code that searches for a generic
libexec (no "kde4" in the pathname) and yet allows KDE to find its
helper tools?

That's my question to this list :-)
Actually, thinking about it, I have one idea:
appending /libexec to every entry in LD_LIBRARY_PATH.
Would that work?

Post by Thiago Macieira

Post by Sune Vuorela
And even write one generic enough to work across distributions.
Debian and friends uses more or less /usr/lib/NAME/libexec
where as fedora and friends uses /usr/libexec/NAME/

What's NAME there? The package name, in order to split up the helper binaries
from different packages?
Do we really need this? After all /usr/bin has no such split.

Post by Thiago Macieira
The big difference there is whether those executables are
architecture-specific in multiarch configurations (e.g.,
/usr/lib64/NAME/libexec). Usually they aren't.

How could an executable not be architecture-specific? I must be missing
something.

[Thiago Macieira]

That's my question to this list :-))
appending /libexec to every entry in LD_LIBRARY_PATH.
Would that work?

That would get you /usr/lib/libexec, which is neither what we have nor what
the FHS recommends.

Post by Thiago Macieira

Post by Sune Vuorela
And even write one generic enough to work across distributions.
Debian and friends uses more or less /usr/lib/NAME/libexec
where as fedora and friends uses /usr/libexec/NAME/

What's NAME there? The package name, in order to split up the helper
binaries from different packages?
Do we really need this? After all /usr/bin has no such split.

Post by Thiago Macieira
The big difference there is whether those executables are
architecture-specific in multiarch configurations (e.g.,
/usr/lib64/NAME/libexec). Usually they aren't.

How could an executable not be architecture-specific? I must be missing
something.

I meant if a 64-bit library (/usr/lib64/libfoo) can use a 32-bit helper
application (/usr/lib/foo/libexec/bar). Or to put it simply: where should the
64-bit helper be installed: /usr/lib or /usr/lib64?

[C***@csiro.au]

On 18/09/2011, at 9:48 AM, Thiago Macieira wrote:

The big difference there is whether those executables are
architecture-specific in multiarch configurations (e.g.,
/usr/lib64/NAME/libexec). Usually they aren't.

How could an executable not be architecture-specific? I must be missing
something.

I meant if a 64-bit library (/usr/lib64/libfoo) can use a 32-bit helper
application (/usr/lib/foo/libexec/bar). Or to put it simply: where should the
64-bit helper be installed: /usr/lib or /usr/lib64?


Since this appears to be a linux-specific question, the lsb-***@lists.linux-foundation.org<mailto:lsb-***@lists.linux-foundation.org> mailing list might be the right place to ask that question. I think there was some discussion on there earlier this year about what should be done around multi-arch in the LSB, so it would seem wise for Qt to be on the same page with the LSB spec and development of upcoming versions. Perhaps iyou could post your question there with some more context - they should be able to recommend the right approach.

PS: The LSB guys recently took on discussion of the FHS too with a view to updating it, so now is a good time to be talking to them about issues like this.

--
Dr Craig Scott
Computational Software Engineering Team Leader, CSIRO (CMIS)
Melbourne, Australia

[David Faure]

Post by Thiago Macieira

That's my question to this list :-))
appending /libexec to every entry in LD_LIBRARY_PATH.
Would that work?

That would get you /usr/lib/libexec, which is neither what we have nor what
the FHS recommends.

Well OK, appending /Qt/libexec to every entry in LD_LIBRARY_PATH (plus default
library paths like /usr/lib or /usr/lib64 somehow...)

Post by Thiago Macieira
I meant if a 64-bit library (/usr/lib64/libfoo) can use a 32-bit helper
application (/usr/lib/foo/libexec/bar). Or to put it simply: where should
the 64-bit helper be installed: /usr/lib or /usr/lib64?

I would say it should be in the same place as the library. But that in turn
leads to the question of how to know where the library is :-)

I am a bit surprised that all this isn't already specified anywhere. Well,
probably because individual apps just like into their install prefix, while
we're aiming for something more generic here.

[Harri Porten]

Post by David Faure
Well OK, appending /Qt/libexec to every entry in LD_LIBRARY_PATH (plus default
library paths like /usr/lib or /usr/lib64 somehow...)

Somehow I feel a bit uneasy about a global search such as this. See below.

Post by David Faure

Post by Thiago Macieira
I meant if a 64-bit library (/usr/lib64/libfoo) can use a 32-bit helper
application (/usr/lib/foo/libexec/bar). Or to put it simply: where should
the 64-bit helper be installed: /usr/lib or /usr/lib64?

I would say it should be in the same place as the library. But that in turn
leads to the question of how to know where the library is :-)
I am a bit surprised that all this isn't already specified anywhere. Well,
probably because individual apps just like into their install prefix, while
we're aiming for something more generic here.

Do we have to? Isn't the "individual apps" part exactly what makes up
libexec? Shouldn't the search be tied to an application - or rather
library? Instead of a global search it'd be a search based on a hint for
a relative offset.

Harri.

[Sune Vuorela]

Post by Harri Porten
Do we have to? Isn't the "individual apps" part exactly what makes up
libexec? Shouldn't the search be tied to an application - or rather
library? Instead of a global search it'd be a search based on a hint for

I'm also started to wonder why we need a 'global' libexec dir. As
libexec is supposed to be implementation details of a specific library,
that specific library can just at build time hardcode the paths to its
libexec parts.

I don't see that library1 should be using library2's libexec content
directly (or vice versa)

OR maybe I missed something?

/Sune

[Sune Vuorela]

Post by David Faure

Post by Sune Vuorela
Debian and friends uses more or less /usr/lib/NAME/libexec
where as fedora and friends uses /usr/libexec/NAME/

What's NAME there? The package name, in order to split up the helper binaries
from different packages?
Do we really need this? After all /usr/bin has no such split.

NAME there could be package name, but it could also be all sorts of
other things, like 'kde4'.

And yes, we need such a split to be sure to avoid clashes. Then several
libraries can have their own 'domagic' helper executable.

We have enough issues with avoiding clashes within /usr/bin.

/Sune

[David Faure]

Post by Sune Vuorela

Post by David Faure

Post by Sune Vuorela
Debian and friends uses more or less /usr/lib/NAME/libexec
where as fedora and friends uses /usr/libexec/NAME/

What's NAME there? The package name, in order to split up the helper
binaries from different packages?
Do we really need this? After all /usr/bin has no such split.

NAME there could be package name, but it could also be all sorts of
other things, like 'kde4'.
And yes, we need such a split to be sure to avoid clashes. Then several
libraries can have their own 'domagic' helper executable.
We have enough issues with avoiding clashes within /usr/bin.

Right. So would NAME=Qt be enough, or would that still lead to too many clash
risks, between qt apps/frameworks? I guess there could be a
QStringList QStandardPaths::libExecPaths(const QString &packageName =
QString())
(whose result could be passed as argument to findExecutable), but that seems
slightly overkill to me (compared to simply finding libexec binaries inside
findExecutable, without a specific API for it).

[Thiago Macieira]

Post by David Faure
Right. So would NAME=Qt be enough, or would that still lead to too many
clash risks, between qt apps/frameworks? I guess there could be a
QStringList QStandardPaths::libExecPaths(const QString &packageName =
QString())
(whose result could be passed as argument to findExecutable), but that seems
slightly overkill to me (compared to simply finding libexec binaries inside
findExecutable, without a specific API for it).

/usr/libexec/OrganisationName/ApplicationName
just like /usr/share/OrganisationName/ApplicationName

[David Faure]

Post by Thiago Macieira
/usr/libexec/OrganisationName/ApplicationName
just like /usr/share/OrganisationName/ApplicationName

Typical view that on top of Qt, there are only standalone applications.

It's time for Qt developers to realize that there can also be frameworks, or
more generally sets of integrated applications, on top of Qt. For instance
this is why I added a "GenericDataLocation" which returns /usr/share, so that
apps can actually share some stuff in there (by using the same subdir name),
instead of each being confined to their own private space.

Back to libexec: it's the same issue. Let's take KDE frameworks as an example,
since you know it. If a library class (KRun) wants to find "kdesu", it can't
be looking into the app-specific
/usr/libexec/OrganisationName/ApplicationName.

Harri and Sune: I agree, we don't need to put all helpers into the same dir,
but we do need a way to find "kdesu", "kioexec", "drkonqi", and so on from any
kde application for instance, so it's not just about per-application stuff
either. The "base directory" has to be global, with a framework name added to
it for instance.

So, I guess the lookup call has to specify the directory.
Maybe like
findExecutable("kdesu", "PREFIX/lib/kdesu/libexec")
but that makes the installation directory impossible to move afterwards, if
the prefix gets hardcoded into the library (I guess that's bad).
So it would be better to specify "kdesu/libexec" and let Qt look into known
prefixes, but indeed PATH and LD_LIBRARY_PATH don't really help, and a new var
sounds overkill...

OK, we can of course do this in KDE's KStandardDirs (returning the list of
paths for the calls to QStandardPaths::findExecutable to look into), i.e.
based on $KDEDIRS plus the hardcoded install prefix.

I was just hoping that a more generic solution is found in Qt, so that any lib
could install a helper binary and find it at runtime.

I guess that's just not possible, except with the compiled-in install prefix
as above.

[Harri Porten]

Post by David Faure
Harri and Sune: I agree, we don't need to put all helpers into the same dir,
but we do need a way to find "kdesu", "kioexec", "drkonqi", and so on from any
kde application for instance, so it's not just about per-application stuff
either. The "base directory" has to be global, with a framework name added to
it for instance.

I mentioned both apps and libraries as a potential hint. The framework's
library would be able to locate the executable relative to its own
installation directory. Or are you foreseeing/knowing about cases where
the search would span various library installation paths?

Harri.

[Oswald Buddenhagen]

Post by David Faure
we do need a way to find "kdesu", "kioexec", "drkonqi",

i'll just point out that these examples are flawed as arguments for
shared locations:
- kdesu is in libexec for plain wrong reasons. it should have stayed in
bin in the first place
- kioexec and drkonqi are implementation details of the frameworks, so
it is perfectly ok to store them in paths which are private to the
frameworks

[C***@csiro.au]

Post by Thiago Macieira

Post by David Faure
Right. So would NAME=Qt be enough, or would that still lead to too many
clash risks, between qt apps/frameworks? I guess there could be a
QStringList QStandardPaths::libExecPaths(const QString &packageName =
QString())
(whose result could be passed as argument to findExecutable), but that seems
slightly overkill to me (compared to simply finding libexec binaries inside
findExecutable, without a specific API for it).

/usr/libexec/OrganisationName/ApplicationName
just like /usr/share/OrganisationName/ApplicationName

If the application is provided as part of the linux distribution, then those would be fine. If not, then the LSB recommends /opt instead of /usr and it also recommends a similar structure below /opt. The LSB recommends (requires?) conformant packages to be installed into their own private area, distinguished by either a LANANA name or your company's domain (if you are an individual and not a company, your options are not so clearly defined!). Thus, the base of your package or suite of packages might be something like one of these:

/opt/kde.org
/opt/kde4

The latter would only be valid if kde4 was a registered LANANA name. Below this base directory, you can use whatever directory structure you want. A recommended approach is to give each application package below this point its own subdirectory, such as:

/opt/kde4/konqueror
/opt/kde4/kmail

This gives each application its own private area, but at a known relative position to other packages in the suite. And this is where this structure helps you in this discussion thread. If you are looking for some kde helper app, it should be at a known relative location to the current executable. This should be true for any app that is notionally part of the same suite (ie from the same "company", or in the examples used here, the same KDE installation). Where a package is included directly in /usr by a distribution, it should still obey this arrangement of keeping all the components relative to each other if that's what the package requires. Unfortunately, under unix people have generally become so accustomed to dumping binaries directly in /usr/bin and libraries in /usr/lib (or /usr/lib6
4), this idea of giving apps their own base directory probably feels foreign (but it is exactly what Mac and Windows do).

So my suggestion would be to assume that your application suite has some common base and then make your searches for helper binaries that come as part of that suite search those relative locations, then fall back to the relevant environment variable depending on what you are looking for (PATH or LD_LIBRARY_PATH). It's not perfect, but I don't think you'll find a way that works in all cases and at least this approach (a) follows LSB-recommended practices, and (b) should teach developers better habits. I suspect that distribution maintainers won't appreciate having app developers dictate directory structure like this though.


--
Dr Craig Scott
Computational Software Engineering Team Leader, CSIRO (CMIS)
Melbourne, Australia

[Thiago Macieira]

Post by Thiago Macieira
Tell me this: how would you write code that searches for a generic libexec
(no "kde4" in the pathname) and yet allows KDE to find its helper tools?

I think this thread has gone off in the wrong direction. Let's go back to
basics: if these helper tools are used by libraries, they cannot depend on the
application's settings.

So I don't see a way to provide anything beyond "system libexec"
(/usr/libexec:/usr/local/libexec) and also allow the user of QStandardPaths to
specify his/her own search paths.