[erlang-patches] customized ERL_NIF

Discussion:

[erlang-patches] customized ERL_NIF_TERM type support

unknown

2014-02-07 17:34:14 UTC

When writing NIFs in C++, it is impossible to employ C++ function
overloading because the underlying type of ERL_NIF_TERM is "unsigned int".
For example:

// won't compile :(
#include <erl_nif.h>
void my_func(ERL_NIF_TERM a) {...}
void my_func(unsigned int a) {...}

This patch allows NIF authors to mutate the type of ERL_NIF_TERM by
defining the macro CUSTOM_NIF_TERM_TYPE(). In the example below, the
underlying unsigned integer type gets wrapped as a C++11 typed
enumeration. The type of ERL_NIF_TERM is now unique and can be used in
overloaded functions.

// compiles! :)
#define CUSTOM_NIF_TERM_TYPE(Type) enum class ERL_NIF_TERM : Type {};
#include <erl_nif.h>
void my_func(ERL_NIF_TERM a) {...}
void my_func(unsigned int a) {...}

The patch has no impact on erl_nif.h if CUSTOM_NIF_TERM_TYPE is not defined
(other than flipping the definition order of ERL_NIF_TERM and ERL_NIF_UINT).

A similar approach has been used on my C++11 NIF wrapper (
https://github.com/goertzenator/nifpp). The wrapper requires manual
installation of a similar patch, and I would love to remove that
requirement.

Regards,
Dan.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20140207/4dbb28c9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: custom_erl_nif_term.patch
Type: text/x-patch
Size: 889 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20140207/4dbb28c9/attachment.bin>

unknown

2014-02-07 18:46:20 UTC

Permalink

Thanks for the patch.

I got one objection/question though. My C++ knowledge is a bit rusty but
as I understand it
the "enum class" construct demands that Type is an integer type.

This is currently true for ERL_NIF_TERM but it might not be in the future.
I earlier tried to define ERL_NIF_TERM as

typedef struct { int x; } ERL_NIF_TERM;

to get stronger type checking even for C. I reverted that due to bad
performance
as gcc has different calling conventions for int and struct {int}.
However, future compilers
might be better at passing struct{int} by value in which case we might
want to change
the definition of ERL_NIF_TERM.

What then? Do we ignore CUSTOM_NIF_TERM_TYPE in such a scenario?

/Sverker, Erlang/OTP

Post by unknown
When writing NIFs in C++, it is impossible to employ C++ function
overloading because the underlying type of ERL_NIF_TERM is "unsigned int".
// won't compile :(
#include <erl_nif.h>
void my_func(ERL_NIF_TERM a) {...}
void my_func(unsigned int a) {...}
This patch allows NIF authors to mutate the type of ERL_NIF_TERM by
defining the macro CUSTOM_NIF_TERM_TYPE(). In the example below, the
underlying unsigned integer type gets wrapped as a C++11 typed
enumeration. The type of ERL_NIF_TERM is now unique and can be used in
overloaded functions.
// compiles! :)
#define CUSTOM_NIF_TERM_TYPE(Type) enum class ERL_NIF_TERM : Type {};
#include <erl_nif.h>
void my_func(ERL_NIF_TERM a) {...}
void my_func(unsigned int a) {...}
The patch has no impact on erl_nif.h if CUSTOM_NIF_TERM_TYPE is not defined
(other than flipping the definition order of ERL_NIF_TERM and ERL_NIF_UINT).
A similar approach has been used on my C++11 NIF wrapper (
https://github.com/goertzenator/nifpp). The wrapper requires manual
installation of a similar patch, and I would love to remove that
requirement.
Regards,
Dan.
_______________________________________________
erlang-patches mailing list
erlang-patches
http://erlang.org/mailman/listinfo/erlang-patches

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20140207/c0aeb606/attachment.html>

unknown

2014-02-07 19:21:27 UTC

Permalink

Thank you for your feedback. Maybe the "hook" approach I suggested is too
generic. My original patch from nifpp actually looks like this...

#ifdef CPP11_UNIQUE_TERM_TYPE
enum class ERL_NIF_TERM : ERL_NIF_UINT;
#else
typedef ERL_NIF_UINT ERL_NIF_TERM;
#endif

...and then to use the typed enum in your c++ nif you write...

#define CPP11_UNIQUE_TERM_TYPE
#include <erl_nif.h>

In the future ERL_NIF_TERM can be changed to a struct and the
CPP11_UNIQUE_TERM_TYPE option can just be ignored. What do you think?

Cheers,
Dan.

On Fri, Feb 7, 2014 at 12:46 PM, Sverker Eriksson <

Post by unknown
Thanks for the patch.
I got one objection/question though. My C++ knowledge is a bit rusty but
as I understand it
the "enum class" construct demands that Type is an integer type.
This is currently true for ERL_NIF_TERM but it might not be in the future.
I earlier tried to define ERL_NIF_TERM as
typedef struct { int x; } ERL_NIF_TERM;
to get stronger type checking even for C. I reverted that due to bad
performance
as gcc has different calling conventions for int and struct {int}.
However, future compilers
might be better at passing struct{int} by value in which case we might
want to change
the definition of ERL_NIF_TERM.
What then? Do we ignore CUSTOM_NIF_TERM_TYPE in such a scenario?
/Sverker, Erlang/OTP
When writing NIFs in C++, it is impossible to employ C++ function
overloading because the underlying type of ERL_NIF_TERM is "unsigned int".
// won't compile :(
#include <erl_nif.h>
void my_func(ERL_NIF_TERM a) {...}
void my_func(unsigned int a) {...}
This patch allows NIF authors to mutate the type of ERL_NIF_TERM by
defining the macro CUSTOM_NIF_TERM_TYPE(). In the example below, the
underlying unsigned integer type gets wrapped as a C++11 typed
enumeration. The type of ERL_NIF_TERM is now unique and can be used in
overloaded functions.
// compiles! :)
#define CUSTOM_NIF_TERM_TYPE(Type) enum class ERL_NIF_TERM : Type {};
#include <erl_nif.h>
void my_func(ERL_NIF_TERM a) {...}
void my_func(unsigned int a) {...}
The patch has no impact on erl_nif.h if CUSTOM_NIF_TERM_TYPE is not defined
(other than flipping the definition order of ERL_NIF_TERM and ERL_NIF_UINT).
A similar approach has been used on my C++11 NIF wrapper (https://github.com/goertzenator/nifpp). The wrapper requires manual
installation of a similar patch, and I would love to remove that
requirement.
Regards,
Dan.
_______________________________________________
erlang-patches mailing listerlang-patches://erlang.org/mailman/listinfo/erlang-patches

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20140207/b6fb2e5b/attachment.html>

unknown

2014-02-10 13:09:11 UTC

Permalink

The "hook" solution has an advantage I think. It leaves it up to the
user to define the type and thereby to "make it work".

I actually don't understand how you get this to work. How can you mix
C++ types within a extern "C" declaration?

/Sverker

Post by unknown
Thank you for your feedback. Maybe the "hook" approach I suggested is too
generic. My original patch from nifpp actually looks like this...
#ifdef CPP11_UNIQUE_TERM_TYPE
enum class ERL_NIF_TERM : ERL_NIF_UINT;
#else
typedef ERL_NIF_UINT ERL_NIF_TERM;
#endif
...and then to use the typed enum in your c++ nif you write...
#define CPP11_UNIQUE_TERM_TYPE
#include <erl_nif.h>
In the future ERL_NIF_TERM can be changed to a struct and the
CPP11_UNIQUE_TERM_TYPE option can just be ignored. What do you think?
Cheers,
Dan.
On Fri, Feb 7, 2014 at 12:46 PM, Sverker Eriksson <

unknown

2014-02-10 19:28:43 UTC

Permalink

Regarding C++ types in extern "C": That is a very good question...

I looked into the C++ standard and other documents. The answer is that
extern "C" just changes the default linkage to "C" instead of "C++".
Everything else remains the same, so you can still use the rest of C++
inside an extern "C", but the linkage may impose some limitations (like no
function overloading). "C" linkage names only indicate the name the of the
function but not the parameter or return types like C++ linkage names.

Now that I am a little more educated on linking and calling conventions, I
am uncomfortable with the patches I have suggested. Sure, they work on
linux with gcc, but will they work on MSVC? What about the other less
common unices? I think depending on linker looseness is brittle.

As an alternate idea, I think I can generate inline C++ wrappers for the
whole NIF API and get a unique ERL_NIF_TERM type that way. It's not very
elegant, but everything gets resolved at compile time with no linker
tricks, and the compiler should flatten the wrapper away so that there is
no overhead.

So, if you agree, I'd like to take this patch out back and shoot it. :)

Dan.

On Mon, Feb 10, 2014 at 7:09 AM, Sverker Eriksson <

The "hook" solution has an advantage I think. It leaves it up to the user
to define the type and thereby to "make it work".
I actually don't understand how you get this to work. How can you mix C++
types within a extern "C" declaration?
/Sverker

Post by unknown
I got one objection/question though. My C++ knowledge is a bit rusty but
as I understand it
the "enum class" construct demands that Type is an integer type.
This is currently true for ERL_NIF_TERM but it might not be in the future.
I earlier tried to define ERL_NIF_TERM as
typedef struct { int x; } ERL_NIF_TERM;
to get stronger type checking even for C. I reverted that due to bad
performance
as gcc has different calling conventions for int and struct {int}.
However, future compilers
might be better at passing struct{int} by value in which case we might
want to change
the definition of ERL_NIF_TERM.
What then? Do we ignore CUSTOM_NIF_TERM_TYPE in such a scenario?
/Sverker, Erlang/OTP
When writing NIFs in C++, it is impossible to employ C++ function
overloading because the underlying type of ERL_NIF_TERM is "unsigned int".
// won't compile :(
#include <erl_nif.h>
void my_func(ERL_NIF_TERM a) {...}
void my_func(unsigned int a) {...}
This patch allows NIF authors to mutate the type of ERL_NIF_TERM by
defining the macro CUSTOM_NIF_TERM_TYPE(). In the example below, the
underlying unsigned integer type gets wrapped as a C++11 typed
enumeration. The type of ERL_NIF_TERM is now unique and can be used in
overloaded functions.
// compiles! :)
#define CUSTOM_NIF_TERM_TYPE(Type) enum class ERL_NIF_TERM : Type {};
#include <erl_nif.h>
void my_func(ERL_NIF_TERM a) {...}
void my_func(unsigned int a) {...}
The patch has no impact on erl_nif.h if CUSTOM_NIF_TERM_TYPE is not defined
(other than flipping the definition order of ERL_NIF_TERM and ERL_NIF_UINT).
A similar approach has been used on my C++11 NIF wrapper (
https://github.com/goertzenator/nifpp). The wrapper requires manual
installation of a similar patch, and I would love to remove that
requirement.
Regards,
Dan.
_______________________________________________
erlang-patches mailing listerlang-patches://
erlang.org/mailman/listinfo/erlang-patches

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20140210/21b987c9/attachment.html>

unknown

2014-02-11 11:33:33 UTC

Permalink

I agree. Shoot!

/Sverker

Post by unknown
Regarding C++ types in extern "C": That is a very good question...
I looked into the C++ standard and other documents. The answer is that
extern "C" just changes the default linkage to "C" instead of "C++".
Everything else remains the same, so you can still use the rest of C++
inside an extern "C", but the linkage may impose some limitations (like no
function overloading). "C" linkage names only indicate the name the of the
function but not the parameter or return types like C++ linkage names.
Now that I am a little more educated on linking and calling conventions, I
am uncomfortable with the patches I have suggested. Sure, they work on
linux with gcc, but will they work on MSVC? What about the other less
common unices? I think depending on linker looseness is brittle.
As an alternate idea, I think I can generate inline C++ wrappers for the
whole NIF API and get a unique ERL_NIF_TERM type that way. It's not very
elegant, but everything gets resolved at compile time with no linker
tricks, and the compiler should flatten the wrapper away so that there is
no overhead.
So, if you agree, I'd like to take this patch out back and shoot it. :)
Dan.
On Mon, Feb 10, 2014 at 7:09 AM, Sverker Eriksson <

The "hook" solution has an advantage I think. It leaves it up to the user
to define the type and thereby to "make it work".
I actually don't understand how you get this to work. How can you mix C++
types within a extern "C" declaration?
/Sverker

unknown

2014-02-13 15:37:20 UTC

Permalink

Hi Sverker.

I want to look at the bad performance issue you observed with respect to
passing "struct{int}" as a parameter vs just passing "int". I wrote a tiny
test program to observe the assembly generated by gcc and the code for the
two cases looked identical. Also, some research into calling conventions
hasn't offered me any explanations yet.

Do you have any details you can share that will help me understand when and
why performance degrades?

I want to understand because I am looking at a struct{int} approach as part
of a C++ nif solution, and I am very sensitive about adding bloat.

Thanks,
Dan.

On Fri, Feb 7, 2014 at 12:46 PM, Sverker Eriksson <

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20140213/78825f4e/attachment.html>

unknown

2014-02-13 16:19:44 UTC

Permalink

As I remember, it was the convention of *returning* int vs struct{int}
that differed (at least).
int was returned in a register while struct{int} was stored on stack and
returning
pointer to it in register.

Maybe there is a difference between C and C++ conventions also.

The top commit on this branch was my attempt with a struct:

git fetch https://github.com/sverker/otp.git ERL_NIF_TERM_as_struct

Just did a little test myself now:

struct X {
int b;
};

struct X bar(struct X);
int foo(int x);

int main()
{
struct X a = {0x77};
struct X x = bar(a);
int y = foo(0x88);

return x.b + y;
}

And gcc 4.6.3 uses the same calling conventions for foo and bar (arg in
edi and return value in eax)
on my x86_64 Linux machine.

I don't remember if it was some other architecture that I saw the
difference on.

/Sverker

Post by unknown
Hi Sverker.
I want to look at the bad performance issue you observed with respect to
passing "struct{int}" as a parameter vs just passing "int". I wrote a tiny
test program to observe the assembly generated by gcc and the code for the
two cases looked identical. Also, some research into calling conventions
hasn't offered me any explanations yet.
Do you have any details you can share that will help me understand when and
why performance degrades?
I want to understand because I am looking at a struct{int} approach as part
of a C++ nif solution, and I am very sensitive about adding bloat.
Thanks,
Dan.
On Fri, Feb 7, 2014 at 12:46 PM, Sverker Eriksson <

unknown

2014-02-13 17:18:10 UTC

Permalink

Thank you for the details.

I see that there are gcc code generation options to choose register or
stack returns for short structs (-fpcc-struct-return and
-freg-struct-return). If not specified the platform default is selected,
whatever that may be. "register" returns must be selected for the tests we
are compiling this week, but on your test years ago "stack" must have been
selected.

Anyway, for my use case I am comfortable going ahead with struct{int} usage.

Thanks again for leading me to the answer,
Dan.

On Thu, Feb 13, 2014 at 10:19 AM, Sverker Eriksson <

Post by unknown
As I remember, it was the convention of *returning* int vs struct{int}
that differed (at least).
int was returned in a register while struct{int} was stored on stack and
returning
pointer to it in register.
Maybe there is a difference between C and C++ conventions also.
git fetch https://github.com/sverker/otp.git ERL_NIF_TERM_as_struct
struct X {
int b;
};
struct X bar(struct X);
int foo(int x);
int main()
{
struct X a = {0x77};
struct X x = bar(a);
int y = foo(0x88);
return x.b + y;
}
And gcc 4.6.3 uses the same calling conventions for foo and bar (arg in
edi and return value in eax)
on my x86_64 Linux machine.
I don't remember if it was some other architecture that I saw the
difference on.
/Sverker

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20140213/4e3320b7/attachment.html>