Interface with R from Lua

Whenever the luajr R package opens a new Lua state, it opens all the standard Lua libraries in that state, as well as the luajr module, which is designed to provide some key capabilities for your Lua code to interface well with R code. This vignette documents the luajr module.

Vector types

The vector types are modelled after C++’s std::vector class. A vector type is what you end up with if you pass an argument into a Lua function using lua_func() with arg code "v", but you can also create new vectors within Lua.

Like their C++ counterparts, vector types all maintain an internal “capacity” which is equal to or greater than their “length”, or actual number of elements. The exception is the character vector, which is implemented internally as a Lua table but has the same interface as the other vector types.

Note that for vector types, indexes start at 1, not at 0. You must be very careful not to access or write out of these bounds, as the luajr module does not do bounds checking. Going out of bounds will cause crashes or other undefined behaviour. Unlike Lua tables, vector types can only be indexed with integers from 1 to the vector length, not with strings or any other types.

Creating and testing vector types

luajr.logical(a, b), luajr.integer(a, b), luajr.numeric(a, b), luajr.character(a, b)

These functions can be used to create “vector types” in Lua code. The meaning of a and b depends on their type. For example:

luajr.numeric()     -- Empty numeric vector
luajr.numeric(n, x) -- Size-n numeric vector, all entries equal to x
luajr.numeric(v)    -- Copied from vector v
luajr.numeric(z)    -- Copied from "vector-ish" object z

Above, n is a nonnegative number, x is a boolean, number, string (as appropriate for the vector type) or nil, v is another vector object of the same type, and z is a table, vector type, or reference type.

If x == nil in the (n, x) form of the above functions, then the values of the vector are left uninitialized (except for luajr.character, where the values are set to the empty string).

luajr.is_logical(obj), luajr.is_integer(obj), luajr.is_numeric(obj), luajr.is_character(obj)

Check whether a value obj is one of the corresponding vector types. These return true if obj is of the corresponding type, and false otherwise.

Vector type methods

All the vector types have the following methods:

#v

Returns the length of the vector.

x = v[i], v[i] = x

Get or set the ith element of the vector.

Again: Note that for vector types, indexes start at 1, not at 0. You must be very careful not to access or write out of these bounds, as the luajr module does not do bounds checking. Going out of bounds will cause crashes or other undefined behaviour. Unlike Lua tables, vector types can only be indexed with integers from 1 to the vector length, not with strings or any other types.

pairs(v), ipairs(v)

Use with a Lua for loop to iterate over the values of the vector. For example:

for i,x in pairs(v) do
    print(i, x)
end

Note that for vector types, pairs and ipairs are interchangeable and the same loop above can be written as

for i = 1, #v do
    print(i, v[i])
end

v:assign(a, b)

Assign a new value to the vector; a and b have the same meaning as in the vector type constructors.

v:print()

Print each value of the vector next to its index, each value on a new line.

v:concat(sep)

Returns a string comprised of the elements of the vector converted into strings and concatenated together with sep as a separator. If sep is missing, the default is ",".

v:debug_str()

Returns a compact string representation of the vector, useful mainly for debugging. This contains the length of the vector, then the capacity of the vector, then each of the vector elements separated by commas.

v:reserve(n)

If n is larger than the vector’s current capacity, suggests that the vector be enlarged to a capacity of n. Otherwise, does nothing.

v:capacity()

Returns the capacity of the vector.

v:shrink_to_fit()

If the vector’s capacity is larger than its length, reallocates the vector so that its capacity is equal to its length.

v:clear()

Sets the size of the vector to 0.

v:resize(n, val)

Sets the size of the vector to n. If n is smaller than the vector’s current length, removes elements at the end of the vector. If n is larger than the vector’s current length, adds new elements at the end equal to val. val can be nil or missing.

v:push_back(val)

Adds val to the end of the vector.

v:pop_back()

Removes one element from the end of the vector.

v:insert(i, a, b)

Inserts new elements before position i, which must be between 1 and #v + 1. a and b have the same meaning as in the vector type constructors.

v:erase(first, last)

Removes elements from position first to position last, inclusive (e.g. so that v:erase(1, #v) erases the whole vector). If last is nil or missing, just erases the single element at position first.

Reference types

The reference types are similar to the vector types, but they are more “low-level” and hence are missing much of the functionality of the vector types. They map directly onto memory managed by R, so judicious use of reference types when passing to or returning from Lua can avoid unnecessary memory allocations and copies.

Although reference types are missing lots of methods that the vector types have, they do have one feature that the vector types don’t have, namely that you can set and get the attributes associated with the underlying R value.

A reference type is what you end up with if you pass an argument into a Lua function using lua_func() with arg code "r", but you can also create new reference types within Lua.

As with the vector types, indexes start at 1, not at 0, and you must be very careful not to access or write out of these bounds.

Finally, because reference types depend upon R, they are not completely thread safe and therefore they cannot safely be used with lua_parallel().

Creating and testing reference types

luajr.logical_r(a, b), luajr.integer_r(a, b), luajr.numeric_r(a, b), luajr.character_r(a, b)

These functions can be used to create “vector types” in Lua code. The meaning of a and b depends on their type. Namely:

luajr.numeric_r(n, x) -- Size n, all entries equal to x if x ~= nil
luajr.numeric_r(z)    -- Copied from "vector-ish" object z

Above, n is a nonnegative number, x is a boolean, number, or string (as appropriate for the vector type) or nil, and z is a table, vector type, or reference type.

luajr.is_logical_r(obj), luajr.is_integer_r(obj), luajr.is_numeric_r(obj), luajr.is_character_r(obj)

Check whether a value obj is one of the corresponding reference types. These return true if obj is of the corresponding type, and false otherwise.

Reference type methods

All the reference types have the following methods:

#v

Returns the length of the referenced vector.

x = v[i], v[i] = x

Get or set the ith element of the referenced vector.

pairs(v), ipairs(v)

Use with a Lua for loop to iterate over the values of the vector.

x = v(attr), v(attr, x)

Get or set the attribute named attr. The second form, with two arguments, sets the attribute to x. Note that even though you can get and manipulate the names attribute, you cannot access a reference vector’s elements by their names.

List type

A list is a special kind of Lua table that can be indexed either with positive integers or with strings. Unlike a Lua table, a list remembers the order in which you have added elements to it, like an R list does.

Creating and testing lists

luajr.list()

Creates a new, empty list.

luajr.is_list(obj)

Check whether a value obj is a list. Returns true if obj is a list, and false otherwise.

List methods

Lists have the following methods:

#v

Returns the length of the list, including both integer- and string-keyed entries. This differs from the behaviour of the # operator on normal Lua tables, which only report the number of integer-keyed entries.

x = v[i], v[i] = x

Get or set element i of the list. i can be either a positive integer or a string. Note that you cannot write to any integer keys greater than #v + 1.

pairs(v), ipairs(v)

Use with a Lua for loop to iterate over the values of the vector. Unlike with vector and reference types, pairs and ipairs are not exactly the same when applied to lists; the former will provide either string or number keys as you iterate through the list (strings if they are present, numbers otherwise) and the latter will give number keys only. Either one iterates through every element of the list and both iterate through the list in the same order.

x = v(attr), v(attr, x)

Get or set the attribute named attr. The second form, with two arguments, sets the attribute to x. Note that, for a list, the "names" attribute is not a simple vector of names, like in R, but is an associative array linking keys to their indices. For example, for a list with elements a = 1, 2, and c = 3, v("names") is equal to { "a" = 1, "c" = 3 }. However, when a list is returned to R, its "names" attribute has the normal R format.

Note that lists have this interface for setting and getting attributes, but unlike the reference types (which also have this capability), they are not internally managed by R. This means that they are safe to use with lua_parallel(), as long as they don’t contain any reference types.

Data frame and matrix types

There are a handful of additional types based on the above types, but which have some special behaviour when returned to R.

Data frame

A data frame can be created with

df = luajr.dataframe()

This is just a luajr.list with the "class" attribute set to "data.frame". However, when a list with this class gets returned to R, it gets turned into a data frame.

Matrix

A matrix can be created with

df = luajr.matrix_r(nrow, ncol)

This is a special kind of luajr.numeric_r with the "dim" attribute set to luajr.integer_r({ nrow, ncol }). It gets recognized as a matrix when returned to R. However, you can only access elements with a single index that starts at 1 and goes in column-major order. So, for example, for a 2x2 matrix, the top-left element has index 1, bottom-left has index 2, top-right has index 3 and bottom-right has index 4.

Note that it is a reference type.

Data matrix

A data matrix can be created with

dm = luajr.datamatrix_r(nrow, ncol, names)

This is similar to luajr.matrix_r, but it has names as column names. names can be passed in as a Lua table (e.g. { "foo", "bar" }) or as a luajr.character or luajr.character_r object.

Using a data matrix is very slightly faster than using a data frame because only one memory allocation needs to be made, but this difference is on the order of microseconds for normal sized data. Also, if you are just going to convert the returned matrix into a data frame anyway, you lose the speed advantage. So don’t worry about it too much.

The column names can be get and set within Lua using the special attribute name "/matrix/colnames".

Constants

The following constants are defined in the luajr table:

luajr.TRUE
luajr.FALSE
luajr.NA_logical_
luajr.NA_integer_
luajr.NA_real_
luajr.NA_character_
luajr.NULL

Note that Lua’s semantics in dealing with logical types and NA values are very different from R’s semantics. Lua does not “understand” NA values in the same way that R does, and it sees the R logical type as fundamentally an integer, not a boolean.

Specifically, whereas in R you could do the following:

x = c(TRUE, FALSE, NA)
if (x[1]) print("First element of x is TRUE!")

in Lua you have to do this:

x = luajr.logical({ luajr.TRUE, luajr.FALSE, luajr.NA_logical_ })
if x[1] == luajr.TRUE then print("First element of x is TRUE!") end

This is because under the hood, R’s TRUE is defined as the integer 1 (though any nonzero integer besides NA will also test as TRUE), FALSE is defined as 0, and NA (when logical or integer) is defined as a special ‘flag’ value of -2147483648 (i.e. -2^31). However, in Lua, anything other than nil or false evaluates as true, meaning that the following Lua code

x = luajr.logical({ luajr.TRUE, luajr.FALSE, luajr.NA_logical_ })
for i = 1, #x do
    if x[i] then print("Element", i, "of x is TRUE!") end
end

will incorrectly claim that luajr.TRUE, luajr.FALSE, and luajr.NA_logical_ are all “TRUE”. So, instead, explicitly set logical values to either luajr.TRUE, or luajr.FALSE, or luajr.NA_logical_, and explicitly test them against the same values.

Note that the different NA constants are not interchangeable. So, when testing for NA in Lua, you have to check if a value is equal to (==) or not equal to (~=) the corresponding NA constant above, depending on the type of the variable in question.

All of this only applies to the vector and reference types described above. Lua also has its own boolean type (with possible values true and false) which will never compare equal with either luajr.TRUE, luajr.FALSE or any of the NA values. A Lua string can never compare equal with luajr.NA_character_, but conversely a Lua number may sometimes compare equal with luajr.NA_logical_, luajr.NA_integer_, or luajr.NA_real_, so do be careful not to mix Lua types and these R constants.

Finally, luajr.NULL can be used to represent a NULL value, either on its own or as part of a table or luajr.list that gets returned to R. If you pass NULL in to Lua through arg code "s", it will come out as nil in Lua; but if you use arg code "r" or "v", it will come out as luajr.NULL.