The Mathemagix glue mechanism

1.Introduction

New functionality can be added to Mathemagix using the “glue” mechanism. There are two typical types of glue:

1. The user has written a new C++ class or a new C++ template class, and would like to use this class from withing the interpreter.
2. The user has written an interface to an external software which comes with its own language and typesystem. In this case, one would like to have an automatic mechanism for making the functionality of the external software available from within Mathemagix.

In both cases, the user has to write a “glue definition routine”, which declares the functionality provided by the glue, using a standard API. This routine can then be called from some central point (at startup or when loading a dynamically linked glue library) in order to make the glue functionality available from within the current evaluator.

2.Gluing C++ classes

2.1.A simple example

Let us first consider writing a glue definition routine for a simple C++ class named color. Assume also that the color class comes with two routines which we wish to export to Mathemagix:

Then the glue definition routine for color typically looks as follows:

The define_type instruction exports the type color to Mathemagix under the name Color. The two next lines export the routines named_color and invert_color under the same names. The type of the routines named_color and invert_color is automatically inferred by the gluing mechanism. The always template parameter is not really important for our simple example; below, we will see how to use it for the conditional definition of glue in the case of template types.

Whenever a new type is exported to Mathemagix, a few standard routines are required to be implemented for this type. In the case of our class color, the following routines must be provided:

The first routine computes a hash value for c, where nat stands for unsigned int. The second flattening routine converts c to a generic expression; this routine will be used when printing a color. The last two routines implement equality testing. Notice that there are several types of equality in Mathemagix. In addition to the == and != operators, you may wish to define

The routine hard_eq is a fast test whether c1 and c2 are represented in the same way in memory. In the case of pointer objects like vectors, one typically tests whether the pointers match; in particular, two identical vectors which are stored at different locations would not be “hard equal”. The routine eq tests whether c1 and c2 are syntactically equal. Typically, the rational number 2/1 might be equal to the integer 1, without being syntactically equal.

2.2.Adding a template class

Let us now consider gluing a template class, such as list<C>. In this case, one typically has to write two glue definition routines. The first one is a template which might look as follows:

The first template parameter Cond is used for defining glue under certain conditions only. A typical use of this parameter is the last routine, which defines the sum of the elements of a list if and only if the type C admits an addition (more explanations on operator classes add_op will be given below). In absence of the condition, instanstation of define_list<C> for a type C without an addition, would try to instantiate sum<C>, leading to a compilation error.

Since C++ only allows us to instantiate the template define_list at compile time, we also need a second glue definition routine define_standard_list which performs the instantiation for a small number of frequently used types. This routine might look as follows:

Most template glue definition routines are in particular instantiated for the generic class of symbolic expressions. In addition, it can be useful to instantiate for a few other classes for which performance might be an issue. Notice that the define_list template should be distributed with the header files of your package: another library might which to instantiate the template for additional classes.

2.3.A short reference guide

The main routines for defining new types and routines are specified in glue.hpp. Here follows a short description:

void define_type<Cond,C> (const generic& name)

void define_constant<Cond,C> (const generic& name, const C& x)

void define_constructor<Cond,C> (generic (*fun) (const C&))

void define<Cond,D> (const generic& name, D (*fun) ())
void define<Cond,D,S1> (const generic& name, D (*fun) (const S1&))
void define<Cond,D,S1,S2> (const generic& name, D (*fun) (const S1&, const S2&))
...

void define_from_iterator<Cond,C> (const generic& name)

void define_caster<Cond,D,S> ([D (*fun) (const S&) [, nat penalty ]])
void define_upgrader<Cond,D,S> ([D (*fun) (const S&) [, nat penalty ]])
void define_downgrader<Cond,D,S> ([D (*fun) (const S&) [, nat penalty ]])
void define_converter<Cond,D,S> ([D (*fun) (const S&) [, nat penalty ]])

void define_primitive<Cond> (const generic& name, generic (*f) (const generic&))

2.4.Implicitly glued classes

Whenever a new type C is defined by the user, a few other related types are added automatically. More precisely, Mathemagix will automatically define the following types:

alias<C>

tuple<C>

alias<tuple<C> >

Remark 1. Concerning C++ references types, one should notice another subtlety when writing accessors for compound data types. For instance, consider the methods

The overloading will allow for both read-access and write-access using the same syntax. However, it sometimes happens that you have a (non constant) table which corresponds to a global environment. In that case, any access to this table will be a write-access independently if you really perform some modifications of the table. This may lead to subtle errors if you really wanted to perform a read-access, since a write-access might actually modify the table (allocating a non existant key-value pair, for instance). This subtlety does not occur for the Mathemagix Alias<C> type.

2.5.Implicit conversions

The routines define_caster, define_upgrader, define_downgrader and define_converter allow the user to define automatic converters between different types. This facility is quite powerful, but has to be used with care: since automatic converters are applied in a very systematic way, they may even be applied in sitations which the user did not foresee.

First of all, the user has to carefully select between the four different types of converters: casters, upgraders, downgraders and converters. These types differ in the way they may be composed: casters are neither left nor right composable, upgraders are right composable, downgraders are left composable and converters are both left and right composable. Given a left composable converter B->C and an arbitrary converter A->B, Mathemagix automatically adds a converter A->C (which is left composable if A->B is left composable and right composable if B->C is right composable). Similarly, if A->B is right composable and B->C is arbitrary, then we add a converter A->C.

Typically, upgraders correspond to type constructors, such as Integer -> Rational or C -> Matrix(C). Similarly, downgraders correspond to type inheritance, such as Rectangle -> Shape, Sum_series(C) -> Series(C), etc. Casters are often used for converters which may involve some loss of data, such as Integer->Double. Left and right composable converters are rarely needed.

A second important property of a converter is the correponding penalty: when several conversion schemes can be used in order to apply a function to some arguments, the scheme with the lowest penalty will be preferred (here we notice that the penalty of a conversion (A,B)->(C,D) is the maximum of the penalties of the conversions A->C and B->D). Among all possible schemes with the lowest penalty, the most specialized function will be chosen (a type T is strictly more specialized than U if there exists a converter T->U but no converter U->T). If no conversion schemes can be found to apply the function, then it will be applied symbolically.

Currently, the following penalties are provided:

PENALTY_NONE

This corresponds to an exact match.

PENALTY_AUTOMATIC

This corresponds to the penalty for automatic language-related conversions, such as Alias(C)->C.

PENALTY_INCLUSION

This penalty should be used for conversions T->U, where T may be viewed as a mathematical subset of U. Example: Integer->Rational. This penalty is the default one for conversions T->U when T is different from Generic.

PENALTY_HOMOMORPHISM

This penalty should be used for conversions T->U which can be viewed as mathematical homomorphisms, but not as inclusions. Examples: Integer->Int or, more generally, Integer->Modular(p).

PENALTY_VARIANT

Sometimes, two distinct libraries implement the same or a similar type. In that case, it may be interesting to provide automatic converters between these types (in both ways). Using the higher penalty PENALTY_VARIANT for this kind of conversions will ensure that operations are performed in the library of the types of the arguments, unless an implementation is only available in the other library.

PENALTY_CAST

This penalty should be used for all conversions which, even though convenient for the user, entail some loss of information. Example: Integer->Double.

PENALTY_FALL_BACK

For any type T, this is the penalty of the conversion T->Generic. Hence, if the user provides a generic fall back implementation of an operation, then this will be the penalty for the application of the fall back method.

PENALTY_PROMOTE_GENERIC

Many composite types, such as Complexify(C), come with an inclusion C->Complexify(C). Although it is generally correct to use PENALTY_INCLUSION for the corresponding penalty, many unwanted conversions may arise when C=Generic. For this reason, the default penaly for conversions of the kind Generic->T is the maximal penalty PENALTY_PROMOTE_GENERIC.

Some common sources of bugs when using overloading and automatic conversions are the following:

1. You specified an upgrader Generic->Polynomial(Generic), but addition on generic polynomials can not be applied so as to add one to a polynomial. The point here is that the penalty of the conversion Generic->Polynomial(Generic) should be the maximal penalty PENALTY_PROMOTE_GENERIC, which is larger than the penalty PENALTY_FALL_BACK for the symbolic addition of expressions. The solution to this problem is to implement the following two specialized additions:

       +: (Polynomial (Generic), Generic) -> Polynomial (Generic)
       +: (Generic, Polynomial (Generic)) -> Polynomial (Generic)

   Actually, these operations may be useful for other coefficient types than Generic, since they can usually be implemented in a particularly efficient way.

2. You forgot to define a generic fall back method for some operation. Sometimes, your implementation may rely on the assumption that a given operation foo has no implementation, so that it will be applied symbolically. When providing an implementation of foo for some type, this assumption may suddenly be violated and provoke infinite loops or incorrect results. In that case, you should provide a default symbolic implementation for foo.

3.Categorial properties

3.1.Introduction

Several common operations, such as addition, multiplication, etc., are exported over and over again by many different types. Mathemagix provides two abstractions to work more efficiently with such common operations: operator classes and categories. Categories reflect the usual mathematical categories to which different classes belong. For instance, integer both matches the ring_cat and ordering_cat. These abstractions have the following advantages:

1. Similar operations on compound types can be factored in a single template which takes an operator class as additional parameter. For instance, one might define a template

   template<typename Op, typename C> vector<C> binary_operation (const vector<C>& v1, const vector<C>& v2);

   for a vectorial binary operation which acts componentwise. This allows us to define the addition on vectors as binary_operation<add_op,C>. Notice that the operator classes may come with additional static methods which easy the implementation of templates such as binary_operation.

2. When exporting a standard operator to the glue, one may use simplified instructions, such as

   define_binary<always,mul_op,string> ();

   Categories allow for similar simplifications:

   define_ring<always,integer> (); define_ordering<always,integer> ();

3. Categorial properties can be used for conditional glue definitions:

   define_ordering< has<C,ordering_cat>, vector<C> > ();

   Notice that a class which matches ring_cat is assumed to correspond to a mathematical ring. For instance, the addition should be commutative, etc. This rule is not strict, however, since it is convenient to regard the class double as a ring. Notice also that a class which matches both ring_cat and ordering_cat is not necessarily an ordered ring. In the future, we might added categories such as ordered_ring_cat if needed.

3.2.Operator and category based glue definitions

Operator and category based routines for glue definition can be found in category_glue.hpp. Here follows a short description of the provided functionality.

void define_unary<Cond,C,Op> ()

void define_binary<Cond,C,Op> ()

void define_test<Cond,C,Op> ()

The standard operator types are defined in operator.hpp. Each operator in particular knows about its name; therefore, the above routines need not provide an additional name argument. Glue definitions for standard categories are all of the form

void define_name<Cond,C> ()

The names of standard categories can be found in category_glue.hpp. Some frequently used categories are the following:

ring

The category of rings, with operations + ,−,× as well as unary - .

euclidean_ring

The category of euclidean rings, with additional operations quo and rem.

field

An euclidean ring which is actual a field and provides the additional operations /.

elementary

A domain on which the elementary functions sqrt,pow,exp,log,cos,sin,tan,acos,asin and atan are defined.

ordering

An ordered domain (but not necessarily totally ordered) with operations ⩽,<,⩾,>.

3.3.Definition of categorial meta properties

Besides defining conditional glue routines, it is necessary to the define the categorial meta-properties of classes separately (it should be possible to avoid this double bookkeeping, but this is not easy to program reliably in C++). Categorial meta properties for a type such as integer should be defined in corresponding meta files integer_meta.hpp.

The file category_meta.hpp provides several macros which should be used for the definition of new meta properties. Any sequence of definitions should be enclosed in a block of the following form:

The META_TYPE macro should refer to the current type to which your definitions will apply, e.g. vector<C> or integer. The META_TMPL macro stands for the corresponding template prefix, e.g. template<typename C> in the case of vector<C> and nothing in the case of integer. The optional typename prefix should be typename<> in the case of template types such as and left empty otherwise. The META_COND stands for the current conditions under which the meta definitions will be valid. In the case when different conditions are needed for different parts of the meta definitions, META_COND should be undefined and redefined.

The following meta definitions are available inside #define-#undef blocks of the above kind:

HAS(my_op)
HAS(my_cat)

HAS_CATEGORY()

HAS_CATEGORY_ABOVE_BASE()

4.An alternative scheme for glue definition

So far, we have described how to define glue for new C++ classes or C++ template classes. When gluing other languages, it may be necessary to have a more direct control over the gluing process, by directly calling the glue definition methods of the abstract evaluator.

4.1.Evaluators

Let us start by giving more details on how expressions are evaluated in Mathemagix. The global variable current_ev contains the current evaluator which is used for the evaluation of expressions. All semantic operations (such as + , - , exp, derive, etc.) on instances of the generic type are performed using this current evaluator (some syntactic operations, such as eq, gen, etc. do not depend on the current evaluator). In addition, a generic expression can be explicitly evaluated using eval.

The current evaluator itself is an instance of the abstract evaluator class, whose interface is specified in evaluator.hpp. Mathemagix provides different evaluators:

• The trivial evaluator evaluates any expression f(x₁,…,x_n) to itself.
• The output evaluator acts like the trivial evaluator but performs a few additional simplifications. For instance, x + 0, 0 + x, x×1, 1×x and x¹ are all evaluated to x.
• The standard evaluator corresponds to the mmx-light interpreter.

New evaluators can be defined by the user, by proving a new concrete implementation of the virtual methods of the abstract evaluator.

Notice that it can be useful to locally change the current evaluator. This can be done using the following mechanism:

For instance, in order to convert an arbitrary generic to an expression generic which can be printed out, we call the routine flatten using the output evaluator. Notice that the same mechaniusm can be used for other expression-based conversions. For instance, in order to transform a sparse polynomial into a dense one, one might construct a “dense polynomial evaluator” which evaluates scalars and the indetermine to dense polynomials and implements the ring operations on dense polynomials. It then suffices to flatten the sparse polynomial using the dense polynomial evaluator.

4.2.Glue definition methods of the abstract evaluator

5.Dynamically linked libraries

A package with a collection of types, routines and glue definition routines should be compiled into a dynamic library, which can then be loaded on the fly into the interpreter. Names of Mathemagix glue libraries should be of the form libmmxname.la or libmmxname.so and the principal glue definition routine for the library should carry the name define_name. Notice that C++ names are mangled, so you may need to declare define_name as a void (*) ().

In the subdirectories examples/fibonacci and examples/gaussian, you may find two simple examples on how to glue new code to Mathemagix. For larger projects, we refer to the documentation on coding conventions and how to add new packages.

5.1.Computing fibonacci numbers

Assume that we want to add a routine fibonacci for computing Fibonacci numbers to the glue. The file fibonnacci.cpp with the routine fibonacci and the corresponding glue definition routine would typically look at follows:

The very last line is added to prevent name mangling of define_fibonacci. We may now compile the file fibonnacci.cpp into a shared library libmmxfibonacci.so:

g++ –shared ‘basix-config –cppflags –libs‘
    fibonacci.cpp -o libmmxfibonacci.so

After putting the directory which contains libmmxfibonacci.so in your LD_LIBRARY_PATH, you may now use the library from within Mathemagix:

5.2.Complexified numbers

To be completed.

6.Exception handling

It is possible to catch exceptions occurring in glued C++ routines within the Mathemagix interpreter. In order to make this work, you should first configure Mathemagix using the (default) option –enable-exceptions. Next, your glue code may raise exceptions of the type mmx::exception.

In fact, it is better not to directly raise exceptions by yourself, but rather use the convenience macros defined in basix.hpp. Indeed, Mathemagix distinguishes two main kind of exceptions, whose default behaviour can be configured:

• Normal exceptions are the typical exceptions that you want to make visible within the interpreter. Since the interpreter is much slower than the C++ code anyway, it is a good practice to heavily test for erroneous exceptional cases in the glue routines. For instance, when making an array access, one should typically test the bounds. Normal exceptions are enabled using the default –enable-exceptions configuration option. When disabled, they will be replaced by assertions inside the code.
• Low level exceptions are additional checks that you may wish to add in critical parts of the C++ code. For instance, low level C++ array access is performance-critical, since it might be used intensively by other C++ routines. When using the –enable-verify configuration option, you may make add additional checks for such low level routines. Since this may slow down the global performance of the system, this option is disabled by default, and should mainly be used for debugging purposes. Instead of directly gluing low level routines to the editor, we rather recommend to write a small wrapper with the necessary checks for the routine.

The two above types of exceptions both come with their corresponding macros. The following macros should be used for raising exceptions:

ERROR(message)

ASSERT(condition,message)

VERIFY(condition,message)

7.The Mathemagix module system

Still to be written and documented.