![]() |
LIBDAR
|
PresentationThe Libdar library has been built from source code originally located directly in the dar command line application. Libdar provides a complete abstraction for handling Disk ARchive (dar)'s archives. The general operations provided are:
The sample code provided here is solely illustrative and is not guaranteed to compile. More detailed API documentation is contained in the source code and can be compiled to the doc/html directory using Doxygen. |
Let's StartConventionsLanguageDar and libdar are written in
C++, and so is the libdar API. While
written in C++, libdar is easily usable with both C and C++ code.
Access from other languages can be provided by specific bindings. I
would only say that you are welcome to provide the necessary bindings
yourself. :-)
Libdar namespaceAll libdar symbols are defined under the libdar namespace. You can either add the using namespace libdar; line at the beginning of your source files: |
using
namespace libdar; get_version(....); |
or, as shown below, you can explicitly use the namespace in front of libdar objects : |
|
Exceptions or no ExceptionsThe library can be used with or
without exceptions. For each example we will see a sample code for both
methods. To the left is with exceptions, to the right without:
|
|
|
All exceptions used by libdar inherit from the pure virtual class Egeneric. The only method you will need to know about for any exception is the get_message() call, which returns a message string describing the message (in human language). The type of the error is defined by the class of the exception. The possible exception types follow: |
class
libdar::Egeneric |
the parent class of all
exceptions (a pure virtual class) |
class libdar::Ememory |
memory has been exhausted |
class libdar::Ebug |
signals a bug, which is
triggered when reaching some code that should never be executed |
class libdar::Einfinint |
arithmetic error detected when
operating on infinint |
class libdar::Elimitint |
a limitint overflow is detected,
indicating the maximum value of the limitint has been exceeded |
class libdar::Erange |
signals a range error |
class libdar::Edeci |
signals conversion problem
between infinint and string (decimal representation) |
class libdar::Efeature |
a requested feature is not (yet)
implemented |
class libdar::Ehardware |
hardware problem is found |
class libdar::Euser_abort |
signals that the user has
aborted the operation |
class
libdar::Ethread_cancel |
A program has requested the
termination of the current thread while libdar was running |
class libdar::Edata |
an error concerning the treated
data has been encountered |
class libdar::Escript |
the script executed between
slices returned an error code |
class libdar::Elibcall |
signals an error in the
arguments given to a libdar call of the API |
class libdar::Ecompilation |
a requested feature has not been
activated at compilation time |
1 - First we *must* check the libdar version |
catch(libdar::Egeneric & e) |
libdar::U_I maj, med, min;
med <
libdar::LIBDAR_COMPILE_TIME_MEDIUM ) "we are
linking against wrong libdar" << std::endl; |
The get_version() function must
be called for
several reasons :
2 - Let's see the available featuresonce we have called a function of the get_version* function it is possible to access the features activated at compilation time: |
} |
|
You can do what you want with
the
resulting values. It's possible to display the available libdar
features or to terminate if you don't find a desired feature. However,
verifying that features are available is not strictly necessary because
libdar will tell you if an operation you call requires a feature that
has not been activated at compilation time, by throwing an Ecompile
exception (or returning
the LIBDAR_ECOMPILATION error
code if you are not using exceptions).
3 -User interactionThe generic user_interaction classTo be able to report messages to the user and prompt for feedback a special class called user_interaction has been introduced. Simply put, user_interaction is a virtual class which you can derive to provide user interaction (a GUI's graphical interaction, for example). There are four methods whose prototypes you must override:void pause (const std::string &message); this method
is
called by libdar when the library needs a yes or no ("continue" or
"kill")
answer to a question, which is provided by the string message.
The question posed by pause() must be answered by returning
normally (= "true") or throwing a Euser_abort
exception if the user refused the proposition. Don't
worry about throwing an exception in your code; it will be trapped by
libdar if you don't want to manage exceptions, and are using libdar in
the "no exception" method. But if you really don't want to throw
exception from your code see next:
bool pause2(const std::string
&message);This is an alternative method to pause() as seen above. In place of
defining a pause() method in
your inherited class, you can redefine the pause2() method. The only
difference with pause() is
that the user answer to the question is returned by a boolean value,
your code does no more have to throw a Euser_abort exception to say "no".
Note that you must not redefine both
pause() and pause2().
void inherited_warning (const std::string &message); libdar calls
this protected method (through
the
public method named warning())
to display an informational message to the user. It is not
always a warning as the name suggests, but sometimes just normal
information. In API 3.0.x this method did not exist, but the
public warning() method itself was pure virtual and thus needed to be
overwritten. Today, the warning()
method is no more pure virtual nor it is even virtual, so the user
defined implementation of message display has to be done in the inherited_warning() method.
std::string get_string (const std::string &message, bool echo); This call is
used
to get an arbitrary answer from the user. This is mainly used to get a
password from the user (when no password has been supplied for an
encrypted archive), the echo argument
indicates if the user response should be displayed back on the screen
(again, very useful for handling password input). If echo is set to "false" the
implementation of get_string() should
hide the characters typed by the user.
user_interaction * clone () const; A deep copy operation must be implemented here. This is because libdar stores the reference to the user_interaction class as a pointer but may want to keep a complete internal copy at some point. A simple implementation of this method should be something like this (even if you don't want to use exceptions): |
user_interaction
*my_own_class::clone() const |
The callback interaction classAn inherited class from user_interaction called user_interaction_callback
provides an implementation of the user interaction based on callback
functions. This allows you to replace the three interactions methods
(pause, warning and get_string) by three normal functions of your
choice, which must be given to the user_interaction_callback's
constructor. The clone()
method is implemented internally, leaving only the three callback
functions to be implemented. Look at
dar's command line code for a practical example. dar's user interaction code is
implemented using an instance of user_interaction_callback
and three static functions in the module dar_suite/shell_interaction.cpp
Pay attention to the contextual value present in the arguments of theses callback functions : |
(t_win *)(context)->show(x); } |
4 - MasksMask are used to define which
files
will be considered and which will not. Libdar implements masks as
several classes that all inherit from a virtual class that defines the
way masks are used. This root class is the class mask and
provides the is_covered()
method which libdar uses to determine which files are considered. There
are many different basic masks classes you can use to build fairly
complex masks:
|
class libdar::mask |
the generic class, parent of all
masks (a pure virtual class) |
class libdar::bool_mask |
boolean mask, either always true
or false, it matches either all files or no files at all |
class libdar::simple_mask |
matches as done by the shell on
the command
lines (see "man 7 glob") |
class libdar::regular_mask |
matches regular expressions (see
"man 7 regex") |
class libdar::not_mask |
negation of another mask |
class libdar::et_mask |
makes an *AND* operator between
two or more masks |
class libdar::ou_mask |
makes the *OR* operator
between two or more masks |
class lbdar::simple_path_mask |
string matches if it is subdirectory of mask or is a directory
that contains the specified path itself |
class libdar::same_path_mask |
matches if the string is exactly
the
given mask (no wild card expression) |
class
libdar::exclude_dir_mask |
matches if string is the given
string or a sub directory of it |
class libdar::mask_list |
matches a list of files defined
in a given file |
Let's play with some masks : |
// all files will be
elected by this mask libdar::bool_mask m1 = true; // m5 will now match
only files that are selected by both m2 AND m4 libdar::et_mask m5; // Frankly, it's
possible to have masks reference each other! libdar::not_mask m7 = m6; |
Now that you've seen the power
of these masks, you should know that in
libdar there are two masks that are required:
Assuming you
choose for example tmp/A as
argument to fs_root (which
argument is present
when creating an archive, for example), your mask will be used against
strings like "tmp/A/some/file"
. This is true up to libdar version 3.0.x (alias release 2.2.x).
Instead, since libdar 4.0.0 the fs_root
argument is expended to an absolute path, so if in the previous
example, your current directory was /var
your masks will be used against strings like "/var/tmp/A/some/file". Of course
there is no difference between theses two libdar revisions when the fs_root argument is an absolute
path.
An exception is the test operation, which has no fs_root argument (because the operation is not relative to an existing filesystem), however the subtree argument exist to receive a mask for comparing the path of file to include or exclude from the test operation. In this case the situation is as if the fs_root was set to the value "<ROOT>". For example, masks will be compared to <ROOT>/some/file when performing an archive test operation.
5 - Let's create a simple archiveNow that we have seen masks and exceptions let's start the real thing: |
|
// creating an archive is simple; it is just
// we will see this structure a bit further U_16 exception, libdar::archive *my_arch
=
libdar::bool_mask(true),
"", // no script will be
executed between
cf_all,// useless here as we don't make a differential
false, // we are not doing a snapshot of the filesystem
true, // we don't save directories marked as caching false, // w e
are not considering the cache directory
0, // we save all files not only those more
recent
&ret, // this value is returned by
libdar
// if you don't want to have statistics of the exception,
// this gives the status of the call |
When creating an archive, the
created
archive object can be used only as reference for an isolation or for
a differential backups. You cannot use it for restoration, listing, or
comparison, because the underlying file descriptors are opened in
write only mode. An implementation which uses file descriptors in
read-write access is not possible and is not a good idea anyway. Why?
Because, for example, if you want to test the newly created archive,
using the newly created object would make the test rely on information
stored in virtual memory (the archive contents, the data location of a
file, etc.), not on the file archive itself. If some corruption
occurred
in the file you would not notice it.
So to totally complete the
archive creation we must destroy the archive
object we have just created, which will also close any file descriptors
used by the object :
|
|
if(exception != LIBDAR_NOEXCEPT) |
|
libdar::archive(dialog,
|
libdar::open_archive_noexcept( dialog,
exception,// this gives the status of the call f(exception !=
LIBDAR_NOEXCEPT) |
Now that we have opened the archive we can perform any operation on it. Let's thus start by testing the archive coherence: |
false, / / we don't want to see the skipped files
|
dialog,
false, // we don't want to see the skipped files
exception,// this gives the status of the call i f(exception != LIBDAR_NOEXCEPT) |
We have tested the archive, but
have
not yet seen the libdar::statistics variable. It can be used when
creating an archive as well as when testing it. This object
reports the number of files treated, as well as the number files with
errors and the type of error. You can have a look at the API reference
guide concerning the archive class methods, for more information about
the uses of theses different fields. Here is
an example, which relies on the class deci to display the
value of an infinint variable:
|
#include "deci.hpp" |
Note that the use of the class
deci
may throw exceptions (in case of lack of memory, for example), and
there is actually no wrapper available to trap the exceptions that may
be thrown by the class deci.
So you have to protect the code using a
try {} catch {}
statement.
You may have noticed that we used NULL as argument for "progressive_report". This argument must either receive NULL as argument or the address of a real allocated statistics object. This object will be updated by the libdar call and if thread support is enabled will let a concurrent thread reading its value to display the current number of file treated for example. Note that there is a little overhead passing a variable to progressive_report, due to the mutex that need be used to avoid one reading data while it is updated by another thread. Follows a example of use of this progressive report feature: |
libdar::statistics report; |
|
|
dialog, false,
// not a verbose output
exception,// this gives the status of the call i f(exception != LIBDAR_NOEXCEPT) |
By default the library will
complete
the listing by calling the warning()
method of the dialog object
one time for each file listed. The warning text will consist of a
string for each file with the relevant information in columns that
would need to be parsed if individual information was desired. This may
not be appropriate for you and as such there is another way to get
listing information. This requires a simple reimplementation of the user_interaction
object.
The user_interaction class
has a listing() method
which provides separate arguments for each piece of information that
can be displayed:
Technical note:
You may notice that file
type is
not explicitly given as a parameter in the listing method. File type is
available as the first byte of the permissions string. This is standard
POSIX stuff except for an extension: "h" for files hard
linked several times. See man 2 stat for
more information. Note however that in the last arguments of this call,
you may easily know whether a file is a directory or not and whether it
is empty or not.
In the user_interaction class
(a virtual class), the listing()
method is not a pure virtual method, so you are not obliged to
overwrite it, but it has just an empty implementation so it does
nothing.
You understand now that, by default, this method is not used. To
activate it, you must call
set_use_listing(true) protected method and of course you will
have to overwrite the listing()
method to have a less silly behavior:
|
// here follows the definition of our own implementation of void
warning(const std::string & message); |
Now assuming we have
implemented the listing()
method in my_user_interaction
class, calling op_listing()
exactly as we did
before, only replacing the dialog
object by one of the my_user_interaction
class. Then this listing()
method will be called for each file to be listed, in place of the warning() method.
As seen at the beginning of
this
tutorial, there is a child class of user_interaction
based on callback functions which is called user_interaction_callback. The listing() method must also be
activated here. This is done automatically when you give a callback
function to the object, thanks to the set_listing_callback()
method :
|
|
Last point about listing, you
have noticed the second argument is normal,
which produces (if the listing()
method of the given
user_interaction object is not overwritten) a listing like tar would
do. In place of normal
you can use tree
or xml.
Note that for theses two new formats the listing() is never used, so even if
you provide a object which listing() method is overwritten, the archive::op_listing() method will
still use the warning()
method of this user_interaction
object to report the archive contents.
7 bis - Dynamic archive contents listingWell, in the previous chapter,
we saw
how to list the archive contents. You can imagine that when you have a
huge archive this call may take a long time to complete and produce a
long output. If your application uses some graphical components and you
want to have a more interesting way for listing the archive contents,
you would maybe like to have just the first level of the directory tree
and let the user open the subdirectories before listing their contents,
having a sort of iterative archive listing. This would avoid having to
wait for the long listing to complete as well as it would avoid having
to allocate memory for all this graphical components representing each
directories and files, entries that will most of the time would not be
read by the user. This is of course possible (Else I would not write
about it ;-) ).
First step, we need to use the listing() method of the user_interaction() as seen above. Second step, we have to call the get_children_of() method of a given archive class. In the following example, we
will use
the user_interaction_callback
class, but you can use your own inherited class from user_interaction, and its listing() class.
|
void listing_callback(const std::string & flag,
// now, instead of calling op_listing()
method of some_archive giving our dialog object as argument, we can
rather call: |
|
libdar::bool_mask(true),
cf_all, // consider any fields for electing false, // we don't want to see the skipped |
libdar::op_diff_noexcept(dialog,
libdar::bool_mask(true),
cf_all, // comparing any fields for electing we don't want to see the skipped
files exception, // this
gives the i f(exception != LIBDAR_NOEXCEPT)
|
Simple, no? Just a note about the what_to_check argument. It may take several values:
9 - restoring filesRestoration of files is done by calling the op_extract method of class archive. |
libdar::bool_mask(true),
false,//
restore directory structure false,// we don't want to see the skipped files |
dialog, "/tmp",
// where to restore files to
libdar::bool_mask(true),
false,// restore directory structure
cf_all,// consider any valuable field to define
// defines also that we will restore
false,// don't warn if a file to be
false,// don't erase EA present in filesystem
false,// we don't want to see the skipped files
NULL, // no progressive report used
exception,// this gives the status of the call i f(exception != LIBDAR_NOEXCEPT) |
Here the what_to_check
argument serves two roles:
10 - isolating the catalogueOK, I know, catalogue is not an English word
(one would rather write catalog),
but that's the name of the C++ class used in libdar, so we will keep
using it here. Note that you don't have to directly access this
class (if you really don't like French).
Isolating the catalogue creates
a new
archive that only contains the list of files and their attributes
(ownership, dates, size, etc.), but no data and no EA are stored in it.
It is very similar to the same archive one gets if one makes a
differential backup of a filesystem that has not changed since the
creation of a reference archive. The usage is very similar to the
archive creation, but it uses a different constructor that has less
arguments :
|
|
"/tmp", // where is saved the
exception, i f(exception != LIBDAR_NOEXCEPT) |
Now we have two archive
objects. my_arch is a
read-only object
created by the "read" constructor. You can do any operations with it,
like file restoration, file comparison, archive testing, as we have
done in the previous sections. The second archive object is my_cat which is a write only
object. It can only be used as a reference for another backup (a
differential backup) or as a reference for a subsequent catalogue
isolation (which would just clone the already isolated catalogue object
here).
Note that, if desired, an
isolated catalogue can be tested, compared
with the filesystem, and you can even try to restore files from it. But
as there is no data associated with the files contents, dar will not
restore any files from it, of course. So for now we will just destroy
the extracted catalogue object, so that all its file descriptors are
closed:
|
|
if(exception != LIBDAR_NOEXCEPT) |
and we keep the my_arch
object for our last operation:11 - creating a differential backupThis operation is the same as
the
first one we did (archive creation). Perhaps you have noted that an
argument was set to NULL.
Here we will pass it my_arch which
means that my_arch will
become the reference archive for the archive we will create. If we had
not destroyed my_cat above,
we could have used it in place of my_arch
for exactly the same result.
|
false, // we are not doing
a snapshot of the filesystem false, // we are not
considering the cache directory false, // we don't want to
see the skipped files NULL); // no progressive report |
libdar::archive *my_other_arch
= my_arch, //
differential backup
libdar::bool_mask(true),
"", // no script will be
executed between cf_all, //
check all fields when looking for changes since
0, // hourshift is
useless here as we make
false, // we are not doing a snapshot of the filesystem
false, // we are not considering the cache directory
false, // we don't want to see the skipped files
NULL, // no progressive
report exception,
// thisgives the status of the call |
As previously, my_other_arch is
a write only object that we won't need
anymore. So we destroy it:
|
|
if(exception != LIBDAR_NOEXCEPT) |
So, we are
at the end of this first part of the tutorial, where we have
seen the general way to manipulate dar archives like dar command-line
does. But we still have an object we need
to destroy to cleanly release the memory used: |
|
if(exception != LIBDAR_NOEXCEPT) |
For more detailed information
about the API you can build the API
documentation from the source code using Doxygen. 12 - Compilation & Linking
|
> cat my_prog.cpp#include <dar/libdar.h> main() { libdar::get_version(...); ... } > gcc -c my_prog.cpp |
|
> gcc -ldar my_prog.o -o my_prog |
Libdar's different flavorsWell, all the compilation and
linking
steps described above assume you
have a "full" libdar library.
Beside the full (alias infinint) libdar flavor, libdar
also comes in 32 and 64 bits
versions. In theses last ones, in place of internally relying on a
special type (which
is a C++ class called infinint)
to
handle arbitrary large integers, libdar32 relies on 32 bits integers
and
libdar64 relies on 64 bits integers (there are limitations which are
described in doc/LIMITATIONS). But all theses libdar version (infinint,
32bits, 64bits) have the same
interface and must be used the same way, except for compilation and
linking.
Theses different libdar
versions can
coexist on the same system, they
share the same include files. But the MODE macro must be set to 32
or 64 when compiling for linking with libdar32 or libdar64
respectively. The MODE macro defines the way the "class infinint" type is
implemented in libdar, and thus changes the way the libdar headers
files are interpreted by the compiler.
|
>
cat my_prog.cpp#include <dar/libdar.h> main() { libdar::get_version(...); ... } > gcc -c -DMODE=32 my_prog.cpp
> gcc -ldar32 my_prog.o -o my_prog |
and replace 32 by 64 to link
with libdar64. Note that libdar*.pc files are
installed in the $(PREFIX)/lib/pkgconfig
file that should simplify (depending on the point of view) all theses
operations.
For example, if you have all different flavors of libdar installed, the
$(PREFIX)/lib/pkgconfig
dir will contain (among other files) the three
following ones:
Thus, if you want to build your
application with libdar32 for example, you will have to call (assuming
you have pkg-config installed)
|
>
gcc `pkg-config --cflags libdar32`
-c
my_prog.cpp
> gcc `pkg-config --libs libdar32` my_prog.o -o my_prog |
13 - Aborting an OperationIf the POSIX thread support is
available, libdar will be
built in a
thread-safe manner, thus you may have several thread using libdar calls
at the same time. You may then wish to interrupt a given thread. But
aborting a thread form the outside (like sending it a KILL signal) will
most of the time let some memory allocated or even worse can lead to
dead-lock
situation, when the killed thread was in a critical section and had not
got the opportunity to release a mutex.
For that reason, libdar proposes a set
of calls to abort any processing libdar call which is ran by a given
thread.
|
// next is the
thread ID in which we want to have
lidbar call canceled// here for simplicity we don't describe the way the ID has been obtained pthread_t thread_id = 161720; // the most simple call is : libdar::cancel_thread(thread_id); // this will make any libdar call in this thread be canceled immediately // but you can use something a bit more interesting: libdar::cancel_thread(thread_id, false); // this second argument is true for immediate cancellation, // of false for a delayed cancellation, in which case libdar aborts the operation // but produces something usable, for example, if you were backing up something // you get a real usable archive which only contains files saved so far, in place // of having a broken archive which miss a catalogue at the end. Note that this // delayed cancellation needs a bit more time to complete, depending on the // size of the archive under process. |
As seen above, cancellation can
be
very simple. What now succeeds when
you ask for a cancellation this way? Well, an exception of type Ethread_cancel
is thrown. All along his path, memory is released and mutex are freed.
Last, the exception appears to the libdar caller. So, you can catch it
to define a specific comportment. If you don't want to use exceptions a
special returned code is used.
|
try libdar::archive *my_arch = |
U_16 ex; archive *my_arch = |
Some helper routines are
available to
know the cancellation status for a particular thread or to abort a
cancellation process if it has not yet been engaged.
|
pthread_t
tid; // how to know if the thread tid is under cancellation process ? if(libdar::cancel_status(tid)) cout << "thread cancellation is under progress for thread : " << tid << endl; else cout << "no thread cancellation is under progress for thread : " << endl; // how to cancel a pending thread cancellation ? if(libdar::cancel_clear(tid)) cout << "pending thread cancellation has been reset, thread " << tid << " has not been canceled" << endl; else cout << "too late, could not avoid thread cancellation for thread "<< tid << endl; |
Last point, back to the Ethread_cancel exception, this class has two methods you may find useful, when you catch it: |
try{ ... some libdar calls } catch(libdar::Ethread_cancel & e) { if(e.immediate_cancel()) cout << "cancel_thread() has been called with "true" as second argument" << endl; else cout << "cancel_thread() has been called with "false" as second argument" << endl; U64 flag = e.get_flag(); ... do something with the flag variable... } // what is this flag stored in this exception ? // You must consider that the complete definition of cancel_thread() is the following: // void cancel_thread(pthread_t tid, bool immediate = true, U_64 flag = 0); // thus, any argument given in third is passed to the thrown Ethread_cancel exception, // value which can be retrieved thanks to its get_flag() method. The value given to this // flag is not used by libdar itself, it is a facility for user program to have the possibility // to include additional information about the thread cancellation. // supposing the thread cancellation has been invoked by : libdar::cancel_thread(thread_id, true, 19); // then the flag variable in the catch() statement above would have received // the value 19.
|
14 - Dar_manager APIFor more about dar_manager, please read the man page where are described in
detail the available features. Note that for dar_manager there is not
a "without exception" flavor,
your
program
must be able to handle exceptions, which by the way are the same as the
ones describes above.
To get dar_manager features you
need to use the class
database
which is defined in the database.hpp
header file so you first need to include that file. Let's see the
different method of the class database
:
Database object constructionTwo constructor are available: |
#include
<dar/database.hpp> |
So far, this is not much
complicated.
You can build an empty database
from nothing, or load a database to memory from a file using the second
constructor. As you can see over the filename to give in this later
constructor, we need a user_interaction
object to be able to inform the
user of any problem that could be met, and a boolean argument called
"partial":
In all the available methods for class database, some require to load the whole database in the memory while some other only require the database header. Loading just the database header is much faster than loading the whole database of course, and as you guess it requires much less memory. While you can perform any operation with a full loaded database, only a subset of available method will be available with a partially loaded database. If you try a method that requires a completely loaded database, you will get an exception if the object you use has been loaded with "true" as last argument (called "partial") of the constructor, and of course an empty database (built with the first constructor) is a completely loaded database, so you don't have restriction in using a new database object. But now let's see the available method for that class: Database's methods
First we will see methods that work with both partially and completely
loaded databases: |
ThanksI would like to thank Wesley Leggette and Johnathan Burchill for having given their feedback and having done grammar corrections to this document. Regards, Denis Corbin. |