Discussion:
[erlang-patches] [jinterface] add erlang term parse/match/bind features
Dmitriy Kargapolov
2014-12-30 20:56:34 UTC
Permalink
This implements functionality similar to following C functions, which are part of erl_interface application:
- ETERM *erl_format(FormatStr, ...);
- int erl_match(ETERM *Pattern, ETERM *Term);

To acheve this new classes introduced:
* OtpErlangVar - variable placeholder;
* OtpErlangBind - variable values collection;
* OtpErlangParser - "erl_format" parser implementation;
* OtpErlangPattern - pattern abstraction with match/bind functions;

Classes representing composite objects OtpErlangList, OtpErlangTuple, OtpErlangMap and new OtpErlangVar implement interface OtpErlangVarrier defining match and bind functions for these objects.

Class OtpErlangMap reworked to be based on HashMap instead of two separate lists keeping keys and values. This is close to native semantics of maps and makes easier implementing basic map manipulations.

It addition to OtpErlangBind custom user's class may be used as receiver of matched variables values. Java reflection is used to prepare variable value setters during the parse stage. Java doc has more details and examples. Test cases implemented.

git fetch https://github.com/x0id/otp.git jinterface_pattern_matching

Thanks.
Vlad Dumitrescu
2014-12-31 15:27:04 UTC
Permalink
Hi Dmitriy,

Nice implementation! I have something similar that I wanted to submit, but
I like some of your details better.

I have a few comments, after browsing the code just briefly:

- I would prefer to have the OEMap changes as a separate PR, as it's a
separate issue and there are more things to address there, like for
example, OEMap could also implement Map, like OEList implements Iterable
(and might implement List).
- I don't like the name OEVarrier, it doesn't mean anything. Maybe
OEMatcher would be better?
- OEBind might be clearer as OEBinding, as it contains a set of bindings?
- I think there are still some merge issues, like for example OEList:299
where the comment is from the old equals method

best regards,
Vlad


On Tue, Dec 30, 2014 at 9:56 PM, Dmitriy Kargapolov <
Post by Dmitriy Kargapolov
This implements functionality similar to following C functions, which are
- ETERM *erl_format(FormatStr, ...);
- int erl_match(ETERM *Pattern, ETERM *Term);
* OtpErlangVar - variable placeholder;
* OtpErlangBind - variable values collection;
* OtpErlangParser - "erl_format" parser implementation;
* OtpErlangPattern - pattern abstraction with match/bind functions;
Classes representing composite objects OtpErlangList, OtpErlangTuple,
OtpErlangMap and new OtpErlangVar implement interface OtpErlangVarrier
defining match and bind functions for these objects.
Class OtpErlangMap reworked to be based on HashMap instead of two separate
lists keeping keys and values. This is close to native semantics of maps
and makes easier implementing basic map manipulations.
It addition to OtpErlangBind custom user's class may be used as receiver
of matched variables values. Java reflection is used to prepare variable
value setters during the parse stage. Java doc has more details and
examples. Test cases implemented.
git fetch https://github.com/x0id/otp.git jinterface_pattern_matching
Thanks.
_______________________________________________
erlang-patches mailing list
http://erlang.org/mailman/listinfo/erlang-patches
Dmitriy Kargapolov
2014-12-31 16:31:09 UTC
Permalink
Hi Vlad,
Thank you much for your notes.

I agree some names were not good enough, I changed these per your
suggestion.
Old comment removed as well.

As to OtpErangMap class - I was not going to re-implement it initially.
But in order to add match/bind I had to do this. I tried to not add much
new methods, keeping implementation rather "just enough" to work with
maps. You are very welcome to add/change whatever you see reasonable in
separate PR though.

Best Regards and Happy New Year!
- Dmitriy.

git fetch https://github.com/x0id/otp.git jinterface_pattern_matching

https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching
https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching.patch
Post by Vlad Dumitrescu
Hi Dmitriy,
Nice implementation! I have something similar that I wanted to submit,
but I like some of your details better.
- I would prefer to have the OEMap changes as a separate PR, as it's a
separate issue and there are more things to address there, like for
example, OEMap could also implement Map, like OEList implements
Iterable (and might implement List).
- I don't like the name OEVarrier, it doesn't mean anything. Maybe
OEMatcher would be better?
- OEBind might be clearer as OEBinding, as it contains a set of bindings?
- I think there are still some merge issues, like for example
OEList:299 where the comment is from the old equals method
best regards,
Vlad
On Tue, Dec 30, 2014 at 9:56 PM, Dmitriy Kargapolov
This implements functionality similar to following C functions,
- ETERM *erl_format(FormatStr, ...);
- int erl_match(ETERM *Pattern, ETERM *Term);
* OtpErlangVar - variable placeholder;
* OtpErlangBind - variable values collection;
* OtpErlangParser - "erl_format" parser implementation;
* OtpErlangPattern - pattern abstraction with match/bind functions;
Classes representing composite objects OtpErlangList,
OtpErlangTuple, OtpErlangMap and new OtpErlangVar implement
interface OtpErlangVarrier defining match and bind functions for
these objects.
Class OtpErlangMap reworked to be based on HashMap instead of two
separate lists keeping keys and values. This is close to native
semantics of maps and makes easier implementing basic map
manipulations.
It addition to OtpErlangBind custom user's class may be used as
receiver of matched variables values. Java reflection is used to
prepare variable value setters during the parse stage. Java doc
has more details and examples. Test cases implemented.
git fetch https://github.com/x0id/otp.git jinterface_pattern_matching
Thanks.
_______________________________________________
erlang-patches mailing list
http://erlang.org/mailman/listinfo/erlang-patches
Vlad Dumitrescu
2015-01-01 14:12:08 UTC
Permalink
Hi Dmitriy,

I'm still going through your code, but I wonder if you could explain why
equals should be implemented in terms of match? I think these are two
different operations and can be relevant only if checking equals between a
regular term and a pattern, but I can't see any reason to want to do that
because these should never be equal. Or maybe I am missing a fine point?...

I am also wondering if OEObject itself could implement OEMatcher and just
call match/bind recursively instead of checking "instanceof OEMatcher" in
OEList, OEMap and OETuple. It feels easier to understand (less conditionals
in the code).

In my implementation, I did it in a slightly different way, mostly in order
to modify existing classes as little as possible, by using an utility class
to traverse terms and do matching and binding. I also have more advanced
conversion routines between Java and Erlang types (most useful are for
Strings, List<->list and Array<->tuple/binary). Maybe you would like to
check at
https://github.com/vladdu/otp/compare/erlang:master...jinterface_new_api?

Maybe we should try to unify our code first and then submit it to OTP?

Happy New Year and I guess that for you it's even Merry Christmas in a
week, right?
Vlad


On Wed, Dec 31, 2014 at 5:31 PM, Dmitriy Kargapolov <
Post by Dmitriy Kargapolov
Hi Vlad,
Thank you much for your notes.
I agree some names were not good enough, I changed these per your
suggestion.
Old comment removed as well.
As to OtpErangMap class - I was not going to re-implement it initially.
But in order to add match/bind I had to do this. I tried to not add much
new methods, keeping implementation rather "just enough" to work with maps.
You are very welcome to add/change whatever you see reasonable in separate
PR though.
Best Regards and Happy New Year!
- Dmitriy.
git fetch https://github.com/x0id/otp.git jinterface_pattern_matching
https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching
https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching.patch
Hi Dmitriy,
Nice implementation! I have something similar that I wanted to submit,
but I like some of your details better.
- I would prefer to have the OEMap changes as a separate PR, as it's a
separate issue and there are more things to address there, like for
example, OEMap could also implement Map, like OEList implements Iterable
(and might implement List).
- I don't like the name OEVarrier, it doesn't mean anything. Maybe
OEMatcher would be better?
- OEBind might be clearer as OEBinding, as it contains a set of bindings?
- I think there are still some merge issues, like for example OEList:299
where the comment is from the old equals method
best regards,
Vlad
On Tue, Dec 30, 2014 at 9:56 PM, Dmitriy Kargapolov <
Post by Dmitriy Kargapolov
This implements functionality similar to following C functions, which are
- ETERM *erl_format(FormatStr, ...);
- int erl_match(ETERM *Pattern, ETERM *Term);
* OtpErlangVar - variable placeholder;
* OtpErlangBind - variable values collection;
* OtpErlangParser - "erl_format" parser implementation;
* OtpErlangPattern - pattern abstraction with match/bind functions;
Classes representing composite objects OtpErlangList, OtpErlangTuple,
OtpErlangMap and new OtpErlangVar implement interface OtpErlangVarrier
defining match and bind functions for these objects.
Class OtpErlangMap reworked to be based on HashMap instead of two
separate lists keeping keys and values. This is close to native semantics
of maps and makes easier implementing basic map manipulations.
It addition to OtpErlangBind custom user's class may be used as receiver
of matched variables values. Java reflection is used to prepare variable
value setters during the parse stage. Java doc has more details and
examples. Test cases implemented.
git fetch https://github.com/x0id/otp.git jinterface_pattern_matching
Thanks.
_______________________________________________
erlang-patches mailing list
http://erlang.org/mailman/listinfo/erlang-patches
Dmitriy Kargapolov
2015-01-01 19:26:21 UTC
Permalink
Hi Vlad,
While continuing study your code I tried to answer your questions.
Reading this please keep in mind the difference in our approaches. In my
approach existing objects structure is re-used to keep pattern elements.
This was done for the sake of simplicity, to keep code less polluted
with new classes and interfaces. JInterface initially announced as
low-level interface to the Erlang, and I tried to follow this way.
Post by Vlad Dumitrescu
Hi Dmitriy,
I'm still going through your code, but I wonder if you could explain
why equals should be implemented in terms of match? I think these are
two different operations and can be relevant only if checking equals
between a regular term and a pattern, but I can't see any reason to
want to do that because these should never be equal. Or maybe I am
missing a fine point?...
This was done for simplicity, to avoid duplicating the code, especially
when operating on complex data like lists, tuples, maps. Less code -
less errors.
Term considered to be pattern in opposite to regular object if it
contains variable placeholders. In such case it should not be used in
any way but for matching against other regular terms. (Regular term can
be considered as an edge case of pattern with no variables and therefore
can also be used as left side of the match operation).
In the light of above, calling equals method for the pattern(s) does not
make any sense and should not bother anyone. For regular term(s) match
and equals methods carry same semantics.
I know that equals method now has some runtime overheads. Imho this is
reasonable sacrificing. The equals() method is not much important. Java
code must preferably work with Java objects, not Erlang terms. What is
really important is to have effective way of importing Erlang terms
translating them to what is suitable for Java code. So much()
implementation is a priority over equals().
Post by Vlad Dumitrescu
I am also wondering if OEObject itself could implement OEMatcher and
just call match/bind recursively instead of checking "instanceof
OEMatcher" in OEList, OEMap and OETuple. It feels easier to understand
(less conditionals in the code).
In such case match/bin would be methods of all objects, including atoms,
integers, strings etc. This does not make much sense imho. I don’t think
it is reasonable to replace one imperfection by another one.
Post by Vlad Dumitrescu
In my implementation, I did it in a slightly different way, mostly in
order to modify existing classes as little as possible, by using an
utility class to traverse terms and do matching and binding.
Our approaches are fundamentally different. I also tried to keep as much
code untouched as possible, but was reasonable lazy to re-create complex
Erlang terms structure in separate pattern object. My approach is a
compromise. It does require touching the core code. Your approach is
smart way to handle pattern matching without touching base code. It may
be (I guess) made as an extension to core JInterface in separate jar
having JInterface as a dependency. Frankly I didn’t have enough time to
follow your way, my code contains less modifications and additions to
the JInterface and probably took less efforts to implement.
Post by Vlad Dumitrescu
I also have more advanced conversion routines between Java and Erlang
types (most useful are for Strings, List<->list and
Array<->tuple/binary). Maybe you would like to check
at https://github.com/vladdu/otp/compare/erlang:master...jinterface_new_api?
I made for my project additional abstractions for type conversion
including lists of objects and maps. I’m not ready to share this, since
it is not perfect at all. And I don’t see any comprehensive generic way
of mapping Erlang data to/from Java data. Anyway, I think that such
stuff may be a separate library dedicated to type conversions and other
aspects of encode/decode of complex user’s data. JInterface as a
low-level library looks better when free of such additions. It is enough
to automate Erlang term structure matching moving type verification and
conversion to the next layer of abstraction or to user’s code (like
custom binders in my case).
Post by Vlad Dumitrescu
Maybe we should try to unify our code first and then submit it to OTP?
First of all, we (and community) should vote for the core approach,
answering the question: should we intervene into the base code or make
parallel patterns object structure? My opinion is that making parallel
structures is more error-prone and makes maintenance harder due to more
similar code to repeat in different places.
Second (less important though) question would be: how reasonable is to
add fancy data conversion helpers to the base JInterface library.
In the code I submitted there is may be space for improvements in the
OEParser class. If we'd like to bring basic data type
verification/conversions to the code. I would prefer to avoid this
though, at least for the initial version.
Post by Vlad Dumitrescu
Happy New Year and I guess that for you it's even Merry Christmas in a
week, right?
Yes, thank you!
Post by Vlad Dumitrescu
Vlad
On Wed, Dec 31, 2014 at 5:31 PM, Dmitriy Kargapolov
Hi Vlad,
Thank you much for your notes.
I agree some names were not good enough, I changed these per your
suggestion.
Old comment removed as well.
As to OtpErangMap class - I was not going to re-implement it
initially. But in order to add match/bind I had to do this. I
tried to not add much new methods, keeping implementation rather
"just enough" to work with maps. You are very welcome to
add/change whatever you see reasonable in separate PR though.
Best Regards and Happy New Year!
- Dmitriy.
git fetch https://github.com/x0id/otp.git jinterface_pattern_matching
https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching
https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching.patch
Post by Vlad Dumitrescu
Hi Dmitriy,
Nice implementation! I have something similar that I wanted to
submit, but I like some of your details better.
- I would prefer to have the OEMap changes as a separate PR, as
it's a separate issue and there are more things to address there,
like for example, OEMap could also implement Map, like OEList
implements Iterable (and might implement List).
- I don't like the name OEVarrier, it doesn't mean anything.
Maybe OEMatcher would be better?
- OEBind might be clearer as OEBinding, as it contains a set of bindings?
- I think there are still some merge issues, like for example
OEList:299 where the comment is from the old equals method
best regards,
Vlad
On Tue, Dec 30, 2014 at 9:56 PM, Dmitriy Kargapolov
This implements functionality similar to following C
- ETERM *erl_format(FormatStr, ...);
- int erl_match(ETERM *Pattern, ETERM *Term);
* OtpErlangVar - variable placeholder;
* OtpErlangBind - variable values collection;
* OtpErlangParser - "erl_format" parser implementation;
* OtpErlangPattern - pattern abstraction with match/bind functions;
Classes representing composite objects OtpErlangList,
OtpErlangTuple, OtpErlangMap and new OtpErlangVar implement
interface OtpErlangVarrier defining match and bind functions
for these objects.
Class OtpErlangMap reworked to be based on HashMap instead of
two separate lists keeping keys and values. This is close to
native semantics of maps and makes easier implementing basic
map manipulations.
It addition to OtpErlangBind custom user's class may be used
as receiver of matched variables values. Java reflection is
used to prepare variable value setters during the parse
stage. Java doc has more details and examples. Test cases
implemented.
git fetch https://github.com/x0id/otp.git
jinterface_pattern_matching
Thanks.
_______________________________________________
erlang-patches mailing list
http://erlang.org/mailman/listinfo/erlang-patches
Dmitriy Kargapolov
2015-01-18 17:18:47 UTC
Permalink
Hi All,
We couldn't find easy way to merge our implementations at this time.
Instead I submitted PR #589: https://github.com/erlang/otp/pull/589, which
comprises of very small addition to the JInterface library (match() and
bind() generic methods). On top of this base change both approaches can be
implemented (even with no more modifications to the core JInterface
library).
Thanks.

On Thu, Jan 1, 2015 at 2:26 PM, Dmitriy Kargapolov <
Post by Dmitriy Kargapolov
Hi Vlad,
While continuing study your code I tried to answer your questions.
Reading this please keep in mind the difference in our approaches. In my
approach existing objects structure is re-used to keep pattern elements.
This was done for the sake of simplicity, to keep code less polluted
with new classes and interfaces. JInterface initially announced as
low-level interface to the Erlang, and I tried to follow this way.
Post by Vlad Dumitrescu
Hi Dmitriy,
I'm still going through your code, but I wonder if you could explain
why equals should be implemented in terms of match? I think these are
two different operations and can be relevant only if checking equals
between a regular term and a pattern, but I can't see any reason to
want to do that because these should never be equal. Or maybe I am
missing a fine point?...
This was done for simplicity, to avoid duplicating the code, especially
when operating on complex data like lists, tuples, maps. Less code -
less errors.
Term considered to be pattern in opposite to regular object if it
contains variable placeholders. In such case it should not be used in
any way but for matching against other regular terms. (Regular term can
be considered as an edge case of pattern with no variables and therefore
can also be used as left side of the match operation).
In the light of above, calling equals method for the pattern(s) does not
make any sense and should not bother anyone. For regular term(s) match
and equals methods carry same semantics.
I know that equals method now has some runtime overheads. Imho this is
reasonable sacrificing. The equals() method is not much important. Java
code must preferably work with Java objects, not Erlang terms. What is
really important is to have effective way of importing Erlang terms
translating them to what is suitable for Java code. So much()
implementation is a priority over equals().
Post by Vlad Dumitrescu
I am also wondering if OEObject itself could implement OEMatcher and
just call match/bind recursively instead of checking "instanceof
OEMatcher" in OEList, OEMap and OETuple. It feels easier to understand
(less conditionals in the code).
In such case match/bin would be methods of all objects, including atoms,
integers, strings etc. This does not make much sense imho. I don’t think
it is reasonable to replace one imperfection by another one.
Post by Vlad Dumitrescu
In my implementation, I did it in a slightly different way, mostly in
order to modify existing classes as little as possible, by using an
utility class to traverse terms and do matching and binding.
Our approaches are fundamentally different. I also tried to keep as much
code untouched as possible, but was reasonable lazy to re-create complex
Erlang terms structure in separate pattern object. My approach is a
compromise. It does require touching the core code. Your approach is
smart way to handle pattern matching without touching base code. It may
be (I guess) made as an extension to core JInterface in separate jar
having JInterface as a dependency. Frankly I didn’t have enough time to
follow your way, my code contains less modifications and additions to
the JInterface and probably took less efforts to implement.
Post by Vlad Dumitrescu
I also have more advanced conversion routines between Java and Erlang
types (most useful are for Strings, List<->list and
Array<->tuple/binary). Maybe you would like to check
at
https://github.com/vladdu/otp/compare/erlang:master...jinterface_new_api?
I made for my project additional abstractions for type conversion
including lists of objects and maps. I’m not ready to share this, since
it is not perfect at all. And I don’t see any comprehensive generic way
of mapping Erlang data to/from Java data. Anyway, I think that such
stuff may be a separate library dedicated to type conversions and other
aspects of encode/decode of complex user’s data. JInterface as a
low-level library looks better when free of such additions. It is enough
to automate Erlang term structure matching moving type verification and
conversion to the next layer of abstraction or to user’s code (like
custom binders in my case).
Post by Vlad Dumitrescu
Maybe we should try to unify our code first and then submit it to OTP?
First of all, we (and community) should vote for the core approach,
answering the question: should we intervene into the base code or make
parallel patterns object structure? My opinion is that making parallel
structures is more error-prone and makes maintenance harder due to more
similar code to repeat in different places.
Second (less important though) question would be: how reasonable is to
add fancy data conversion helpers to the base JInterface library.
In the code I submitted there is may be space for improvements in the
OEParser class. If we'd like to bring basic data type
verification/conversions to the code. I would prefer to avoid this
though, at least for the initial version.
Post by Vlad Dumitrescu
Happy New Year and I guess that for you it's even Merry Christmas in a
week, right?
Yes, thank you!
Post by Vlad Dumitrescu
Vlad
On Wed, Dec 31, 2014 at 5:31 PM, Dmitriy Kargapolov
Hi Vlad,
Thank you much for your notes.
I agree some names were not good enough, I changed these per your
suggestion.
Old comment removed as well.
As to OtpErangMap class - I was not going to re-implement it
initially. But in order to add match/bind I had to do this. I
tried to not add much new methods, keeping implementation rather
"just enough" to work with maps. You are very welcome to
add/change whatever you see reasonable in separate PR though.
Best Regards and Happy New Year!
- Dmitriy.
git fetch https://github.com/x0id/otp.git
jinterface_pattern_matching
https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching
https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching.patch
Post by Vlad Dumitrescu
Post by Vlad Dumitrescu
Hi Dmitriy,
Nice implementation! I have something similar that I wanted to
submit, but I like some of your details better.
- I would prefer to have the OEMap changes as a separate PR, as
it's a separate issue and there are more things to address there,
like for example, OEMap could also implement Map, like OEList
implements Iterable (and might implement List).
- I don't like the name OEVarrier, it doesn't mean anything.
Maybe OEMatcher would be better?
- OEBind might be clearer as OEBinding, as it contains a set of bindings?
- I think there are still some merge issues, like for example
OEList:299 where the comment is from the old equals method
best regards,
Vlad
On Tue, Dec 30, 2014 at 9:56 PM, Dmitriy Kargapolov
This implements functionality similar to following C
- ETERM *erl_format(FormatStr, ...);
- int erl_match(ETERM *Pattern, ETERM *Term);
* OtpErlangVar - variable placeholder;
* OtpErlangBind - variable values collection;
* OtpErlangParser - "erl_format" parser implementation;
* OtpErlangPattern - pattern abstraction with match/bind functions;
Classes representing composite objects OtpErlangList,
OtpErlangTuple, OtpErlangMap and new OtpErlangVar implement
interface OtpErlangVarrier defining match and bind functions
for these objects.
Class OtpErlangMap reworked to be based on HashMap instead of
two separate lists keeping keys and values. This is close to
native semantics of maps and makes easier implementing basic
map manipulations.
It addition to OtpErlangBind custom user's class may be used
as receiver of matched variables values. Java reflection is
used to prepare variable value setters during the parse
stage. Java doc has more details and examples. Test cases
implemented.
git fetch https://github.com/x0id/otp.git
jinterface_pattern_matching
Thanks.
_______________________________________________
erlang-patches mailing list
http://erlang.org/mailman/listinfo/erlang-patches
Loading...