STLplus C++ Library Collection - Persistence Library
Author: Andy Rushton

# STLplus C++ Library Collection

Website Collection

# Persistence Library

- Browse Header persistence.hpp
- Contents
- Dependencies
- Introduction
- Overview
- Subsystems in the Persistence Library
- Include Files

## Contents

The components provided are:

- persistent_contexts.hpp: Persistence Context Classes
- persistent_basic.hpp: Persistence of Basic Types
- persistent_stl.hpp: Persistence of STL Containers
- persistent_stlplus.hpp: Persistence of STLplus Containers
- persistent_pointers.hpp: Persistence of Pointers
- persistent_shortcuts.hpp: Shortcut Functions
- Examples

## Dependencies

The persistence library has a soft dependency on the containers and portability libraries. This means that it only requires these two libraries in order to make their contents persistent. If there is no need for that functionality, then it is possible to compile the persistence library to exclude them.

To compile the persistence library in stand-alone mode, you need to add the following pre-processor switch to the compiler setup. If you are building using the provided makefile system, then this can be added to the file `persistence/Makefile`.

    CPPFLAGS += -DNO_STLPLUS_CONTAINERS
    CPPFLAGS += -DNO_STLPLUS_INF

In fact, these lines are already present in the makefile, but commented out.

You will also need to provide these switches in any program that uses the persistence library, since it controls what's visible in the headers that are included into your code.

You will also need to comment out the dependencies:

    #LIBRARIES += ../portability
    #LIBRARIES += ../containers

Note the # comment at the start of the line to comment out these dependencies.

## Introduction

Persistence is the ability to dump a data structure to a "serialised" form and then restore it later either in the same run of the program, a later run of a program, or even in a different program. It is an easy way to save a program's state or to communicate information in a structural form between programs.

Persistence is not limited to disk dumps, since the same idea can be used to transfer information from one program to another down a pipe or even an Internet connection. In effect you can use persistence to communicate a data structure of any complexity between two programs even if they are running on different computers under different operating systems.

At a more basic level, persistence does away with the need to design file formats. Instead, just design a data structure and then make that data structure persistent. The file format is designed for you by the persistence subsystem.

The persistent format chosen for the STLplus is a binary format so is extremely efficient both in data size and in CPU time required. For example, text formats such as XML tend to be dominated by the processing required to convert integer values between their machine form (2's-complement binary) and the text form (sign-magnitude decimal). This problem doesn't occur with the persistence format which dumps and restores in the native binary form.

The data format is platform-independent so that there are no problems in transferring data between different computers, regardless of what operating system is running, what compiler was used, what processor is being used, what convention is used to store numeric types (i.e. little-endian versus big-endian) or even what size of word is being used (e.g. 32-bit versus 64-bit).

So, for example, a program compiled with Gnu's gcc on Gnu/Linux running on a 32-bit big-endian processor can transfer data either as a file or over the internet to a program compiled with Microsoft's Visual Studio, running on a 64-bit little-endian processor.

The approach taken with the persistence subsystem is to provide a toolkit which makes it easy if not trivial to make a data structure persistent. However, it is not totally automatic - C++ is too flexible a language to be able to take any data structure and just dump it. This is why the approach has been to provide a toolkit out of which persistence routines can be written.

The toolkit provides a set of functions for dumping and restoring a wide range of types. All the basic C types are made persistent, as are C++ types like std::string and std::complex. However, the real power of the persistence functions is that template functions are provided for making all of the STL and STLplus container classes persistent.

## Overview

The idea is that a container is made persistent by dumping its contents using a dump routine for the contained data type. For example, a std::vector of std::string is dumped by dumping vector-specific information and then repeatedly calling the dump routine for string. The restore function restores the vector and then repeatedly calls the restore function for string to restore the vector's contents.

The same concept is applied to all the container classes. Therefore, to make a container persistent, all you have to do is supply dump and restore functions for the contained type. If the contained type is a basic C or C++ type, then these functions are already provided and the data structure is already persistent.

The core of the persistence system are the persistence contexts, which act like I/O objects. Indeed, the persistence contexts take an IOStream object to perform the I/O part of the dump/restore operation.

The sequence of events is:

1. Create an IOStream device for the source/target of the dump/restore
2. Create a dump_context/restore_context for the dump/restore
3. Use the Persistence Functions to perform the dump/restore

The persistence functions are used to build up layers of recursive function calls which parallel the data structure being made persistent. So for example, a list of vectors of strings will use the persistence functions for list, calling the functions for vector, which in turn calls the functions for string. Two sets of functions are written, one for dumping and one for restoring.

Persistence functions for simple types are non-templates and take two arguments. For example, here are the dump/restore functions for std::string:

    void stlplus::dump_string(stlplus::dump_context&, const std::string& data);

    void stlplus::restore_string(stlplus::restore_context&, std::string& data);

The persistence functions for container templates take an extra parameter - the name of a function to dump/restore the element type of the container. For example, here are the dump/restore functions for vector:

    template<typename T, typename D>
    void stlplus::dump_vector(stlplus::dump_context&, const std::vector<T>& data, D dump_fn);

    template<typename T, typename R>
    void stlplus::restore_vector(stlplus::restore_context&, std::vector<T>& data, R restore_fn);

Note how there is a final parameter dump_fn/restore_fn. In use, these must be provided with the name of the function to dump/restore the element type.

For example, to dump a vector of string, combine the above two functions:

    stlplus::dump_vector(context, data, stlplus::dump_string);

To create a list of vectors of strings, simply add the next layer. The list functions are:

    template<typename T, typename D>
    void stlplus::dump_list(stlplus::dump_context&, const std::list<T>& data, D dump_fn);

    template<typename T, typename R>
    void stlplus::restore_list(stlplus::restore_context&, std::list<T>& data, R restore_fn);

However, the last parameter - the dump_fn and restore_fn - must be only two-parameter functions. This means you need to wrap the existing dump_vector call into a specialisation function:

    void dump_string_vector(stlplus::dump_context&, const std::vector<std::string>& data)
    {
      stlplus::dump_vector(context, data, stlplus::dump_string);
    }

    void restore_string_vector(stlplus::restore_context&, std::vector<std::string>& data)
    {
      stlplus::restore_vector(context, data, stlplus::restore_string);
    }

It is now possible to implement the list routines. For example, to dump a list of vectors of string, combine the above two functions:

    stlplus::dump_list(context, data, dump_string_vector);

And of course, this can then be wrapped into a two-parameter function to be used in yet another level of data structure.

    void dump_string_vector_list(stlplus::dump_context&, const std::list<std::vector<std::string> >& data)
    {
      stlplus::dump_list(context, data, stlplus::dump_string_vector);
    }

    void restore_string_vector_list(stlplus::restore_context&, std::list<std::vector<std::string> >& data)
    {
      stlplus::restore_list(context, data, stlplus::restore_string_vector);
    }

There are also some shortcut functions that perform the whole three step sequence of creating an IOStream device, creating a context and performing the dump/restore in a single step. For example, to dump to a file called data.pst:

    stlplus::dump_to_file(data, "data.pst", dump_string_vector_list, 0);

## Subsystems in the Persistence Library

The rest of the explanation is split up into separate pages for each of the main subsystems within the persistence library:

- persistent_contexts.hpp: Persistence Context Classes
- persistent_basic.hpp: Persistence of Basic Types
- persistent_stl.hpp: Persistence of STL Containers
- persistent_stlplus.hpp: Persistence of STLplus Containers
- persistent_pointers.hpp: Persistence of Pointers
- persistent_shortcuts.hpp: Shortcut Functions
- Examples

## Include Files

This section lists the types that are already made persistent by the built-in functions in the persistence library. In each case, the table below classifies the type and includes a link to the relevant page describing that subset of types. For example, all the basic (c-style) types are described on one page, the STL types on another and so on. Click on the "Library" column of the type you are interested in to go to the page for that type and others in the same category.

Persistence Functions
Type | Library | Include | Function Names
--- | --- | --- | ---
bool | C | persistent_bool.hpp | dump/restore_bool
char | C | persistent_int.hpp | dump/restore_char
signed char | C | persistent_int.hpp | dump/restore_signed_char
unsigned char | C | persistent_int.hpp | dump/restore_unsigned_char
short | C | persistent_int.hpp | dump/restore_short
unsigned short | C | persistent_int.hpp | dump/restore_unsigned_short
int | C | persistent_int.hpp | dump/restore_int
unsigned | C | persistent_int.hpp | dump/restore_unsigned
long | C | persistent_int.hpp | dump/restore_long
unsigned long | C | persistent_int.hpp | dump/restore_unsigned_long
inf | STLplus | persistent_inf.hpp | dump/restore_inf
enum{} | C | persistent_enum.hpp | dump/restore_enum
float | C | persistent_float.hpp | dump/restore_float
double | C | persistent_float.hpp | dump/restore_double
T* (simple) | C/C++ | persistent_pointer.hpp | dump/restore_pointer
T* (polymorphic, interface) | C++ | persistent_interface.hpp | dump/restore_interface
T* (polymorphic, callbacks) | C++ | persistent_callback.hpp | dump/restore_callback
shared_ptr<T> (simple) | STL | persistent_shared_ptr.hpp | dump/restore_shared_ptr
shared_ptr<T> (polymophic, interface) | STL | persistent_shared_ptr.hpp | dump/restore_shared_ptr_interface
shared_ptr<T> (polymorphic, callbacks) | STL | persistent_shared_ptr.hpp | dump/restore_shared_ptr_callback
smart_ptr<T> (simple) | STLplus | persistent_smart_ptr.hpp | dump/restore_smart_ptr
smart_ptr_clone<T> (polymophic, interface) | STLplus | persistent_smart_ptr.hpp | dump/restore_smart_ptr_clone_interface
smart_ptr_clone<T> (polymorphic, callbacks) | STLplus | persistent_smart_ptr.hpp | dump/restore_smart_ptr_clone_callback
simple_ptr<T> (simple) | STLplus | persistent_simple_ptr.hpp | dump/restore_simple_ptr
simple_ptr_clone<T> (polymophic, interface) | STLplus | persistent_simple_ptr.hpp | dump/restore_simple_ptr_clone_interface
simple_ptr_clone<T> (polymorphic, callbacks) | STLplus | persistent_simple_ptr.hpp | dump/restore_simple_ptr_clone_callback
char* | C | persistent_cstring.hpp | dump/restore_cstring
string | STL | persistent_string.hpp | dump/restore_string
basic_string<T> | STL | persistent_string.hpp | dump/restore_basic_string
bitset<N> | STL | persistent_bitset.hpp | dump/restore_bitset
complex<T> | STL | persistent_complex.hpp | dump/restore_complex
deque<T> | STL | persistent_deque.hpp | dump/restore_deque
list<T> | STL | persistent_list.hpp | dump/restore_list
vector<T> | STL | persistent_vector.hpp | dump/restore_vector
pair<T1,T2> | STL | persistent_pair.hpp | dump/restore_pair
triple<T1,T2,T3> | STLplus | persistent_triple.hpp | dump/restore_triple
foursome<T1,T2,T3,T4> | STLplus | persistent_foursome.hpp | dump/restore_foursome
hash<K,T,H,E> | STLplus | persistent_hash.hpp | dump/restore_hash
map<K,T> | STL | persistent_map.hpp | dump/restore_map
multimap<K,T> | STL | persistent_multimap.hpp | dump/restore_multimap
set<T> | STL | persistent_set.hpp | dump/restore_set
multiset<T> | STL | persistent_multiset.hpp | dump/restore_multiset
digraph<N,A> | STLplus | persistent_digraph.hpp | dump/restore_digraph
matrix<T> | STLplus | persistent_matrix.hpp | dump/restore_matrix
ntree<T> | STLplus | persistent_ntree.hpp | dump/restore_ntree