What is the "Lib" structure?

[greengeek]

What is the structure of a "lib"?

Is it full of C code?
Is it an ELF?
Does it include scripts?
Is it some form of sfs or filesystem?
Are they editable?

And are they scannable by antivirals?

If we go on a lib hunt how do we know that any given lib - of the correct name - doesn't contain malware?

[vovchik]

Dear greengeek,

Normal shared libraries in Linux are elf binaries (.so), usually written in c or c++ and compiled, but without the usual "main" bit. They generaly consist of a collection functions that will be called by external programs. In order to use them, you need to tell the program you are writing to import those functions from the lib in question (you need to know their exact names) and to know the types of values that are passed to the functions and those which are returned. And, yes, they can be scanned by virus scanners, too.

I use external libs all the time when programming...

With kind regards,
vovchik

[wiak]

If we go on a lib hunt how do we know that any given lib - of the correct name - doesn't contain malware?

It could contain malware so you need to be able to trust where you get it from, or as vovchik says use tools such as virus scanners on them.

My quick (probably inaccurate, and not guaranteed even correct, but slightly detailed take is):

Libraries are just collections of pre-compiled functions. Like the main application that uses them they are also in Executable and Linking Format (ELF) image format but they are not executable programs per se, rather they are a collection of function object files (that have extension .o), which are compiled bits of code, but not yet 'linked', which roughly means that the address to access them once loaded into memory hasn't been determined yet. Original source code for the functions used doesn't have to be C, can be any compilable programming language, but I guess we are usually talking about C or C++

As you probably know, there are two types of library:

Static libs, whose filenames end with .a
Programs that accesss these use static linking at compile time.

and

shared Dynamic libs, whose filenames end with .so
Programs that access these can either use dynamic linking of shared libraries at run time, or something called Dynamic Loading (see last para of this post).

Whether a program is compiled to access static libs or shared dynamic libs, it needs to include what is called a 'prototype' for each function that is contained in each required lib. Such 'prototypes' declares what vovchik refers to as "the types of values that are passed to the functions and those which are returned". Including prototypes for lib functions is most often done via the program containing an include header file (that contains the function prototypes), which takes the form:

Code: Select all

#include "<header file name>"

Static libs are a collection of object files (.o) that are assembled into an archive (using the ar command) that also get indexed during their construction (index being like a book index to the various object functions). This form of library is identified by its ".a" extension. You can then compile a program to use that library (i.e. incorporate its object functions) and the result will be an program that has that has a copy of that whole static lib code built in. You can compile other programs to also build in that same static library. Problem is that when you execute that program and it needs to access some of the static lib functions the programs whole copy of the lib gets loaded into RAM and if executing a different program which was also compiled to use that same static code that also loads its own copy into RAM. Over time that's a lot of duplicated code in RAM...

Shared Dynamic libs only need one copy of the lib loaded into RAM and programs compiled to use a dynamic lib (or dynamic libs) can all access the functions when required. So that's a form of sharing that avoids duplication of code copied into RAM.

I should also mention that when an application uses shared (dynamic) libs, the actual loading of the lib is done via the kernel itself calling up another ELF shared lib called the dynamic linker whose filename is ld-linux.so (generally stored in /lib). Its job is to basically load the shared lib into RAM, if it is not already there, and fill in the missing addresses so the functions can be accessed by the program (that's called relocation). Such relocation is just a matter of filling in address information in tables that are stored in RAM for that (indirection/offset/pointer) purpose. /lib/ld-linux.so is kind of like /bin/sh, but instead of being an interpreter for shell scripts, /lib/ld-linux.so is an interpreter for ELF images. For example, an ELF image can contain an init section (.init) and a termination function (defined in ELF .fini section) and after relocation completed /lib/ld-linux.so does that initialisation stuff as part of its job and the termination/cleanup stuff when the lib is unloaded.

Lastly, but you can ignore this paragraph really, shared Dynamic libs can alternatively/also be accessed via Dynamic Loading (instead of above described dynamic linking). Its basically the same as dynamic linking except that instead of just having the kernel automatically calling up ld-linux.so to do the loading/linking (via the #include headers), the program itself plays a dynamic role through including specific Application Interface code in it to directly specify a library and function of that library it wants to use (basically it is an API to allow userland program to directly use ld-linux.so). That's a powerful mechanism userland coders can use to dynamically change what library/function they want to use (e.g. via commandline args), rather than just via include headers, but in general you don't need to understand/worry about Dynamic Loading - its just a special facility for programmers that I mention for more completeness.

Any compiled program can be coded to call up executable shell scripts as part of their design of course... in C, that function is provided by system(<filename>) call, which uses system's /bin/sh to execute whatever <filename> is.