h1

Lighting up the tunnErl Pt. 3 – AftErl the basics

January 5, 2009

As promised this post follows on from the previous post, documenting my experience with Erlang over the past few months.

This one focuses on cock-ups and general mistakes I’ve made over the past few months, putting them here helps you to not make the same mistake & helps me to remember what to I did at that time, as I’ve said before killing a few birds with one stone is my thing.

AftErl the basics

By now I had decided to play around with Twitter’s API seeing as I use it a hell of a lot and web based applications are my thing (there is no better way to learn something that to use it). This provided me with quite a bit of insight when it came to developing in Erlang. I found these to be common over sights & issues:

  • Forgetting to end a line properly.
  • State weirdness.
  • Not catching a process message.
  • Spawning processes within processes.
  • Handling stopping child processes.
  • Module definition not the same as file name.
  • Creating result structures.
  • Parsing XML.
  • Working processes die once they are linked to a supervisor.

Forgetting to end a line properly

This was one of my most common mistakes, something that I still miss from time to time, the easiest thing to do is to remember the following:

  1. Each pattern match within a case ends with a semi-colon, unless its the last which is not terminated.
  2. If a case encapsulates multiple cases each end clause should finish with a semi-colon, with the last encapsulated case not being terminated.
  3. When refactoring a case, be sure to take note of the end of the case, moving the last case to the anywhere else means that that and the now last case needs to be refactored.

I assure you you will no doubt get into this issue as a newbie, as I’ve read it is one of the few issues others have with it as beginners. As long as you remember those points and if really stuck check against a piece of code that you know works & you should be fine, just a couple of examples taken from a system I am working on at the moment:

case gb_trees:is_defined(User, State#foobarz.users) of
    false->
        io:format("~p connected to ~p~n", [User,?MODULE]),
        {{ok, "connected"},
          gb_trees:insert(User, {User,From}, State#foobarz.users)};
    true -> {{error, "Unable to connect."},
                       State#foobarz.users}
end.

This is an example of a basic case clause, as we don’t want to process the results, we simply terminate with a full stop. to Tell Erlang to return case’s results.

determine_user_action(GroupName,{Action,PayLoad},UsersList) ->
    case Action of
        drop_group ->
            GroupMsg = "Sending disconnect message to ~s~n",
            send_msg_to_users({drop_group,GroupName},UsersList,GroupMsg);
        receive_msg ->
            case PayLoad of
               {CreatedOn,Sender,Message} ->
                   GroupMsg = "Sending to users ~s~n",
                   send_msg_to_users({receive_msg, CreatedOn,Sender,Message},UsersList,GroupMsg);
               _ ->
                  {error, "Illegal payload format"}
            end;
        _ -> {error, "Illegal action!"}
    end.

This method is a little more involve and displays and example of a case encapsulated method, first we do the basic pattern matching, if we get the atom recieve_msg we want to use another case to check that the payload is what we expect it to be, you main thing to notice here is that the inner case statement terminates with a semi-colon which tells Erlang that this is the end of the expression.

Not catching a process message.

This is another common mistake on my behalf, either passing the wrong pattern or forgetting a trap_exit, this is something to watch out for. to save a lot of unnecessary system crashes, I’d suggest testing out pattern matches via a separate virtual node, allowing you to determine the best patterns for the task. If the process is supervised make sure that you use process_monitor(trap_exit, true) with the init of the gen_server.

State weirdness

Remember when using gb_trees:insert and the such like the new data is returned as the result & not updated internally, it is very easy to forget this, especially coming from an OOP background. So if you notice that your state is not being updated, check that the result of such methods is being passed and not left in the ether.

Another similar issue I have recently come across is mangled states within gen_server, I was basically trying to update a state if something changed, though instead of return the state property if the was no update I passed the whole state, leading in warped data, this can be quite serious & very hard to find. Here is the culprit:

handle_call({disconnect, User}, _From, State) ->
    {Reply,NewTree} =
        case gb_trees:is_defined(User, State#foobarz.users) of
            true ->
                {{ok, "User dropped"},
                 gb_trees:delete(User, State#foobarz.users)};
            false ->
                 {{error, "Unable to drop group."},
                  State}
        end,
{reply, Reply, State#foobarz{ users = NewTree }};

Well it seems safe enough right?, after noticing a bug in part of my system and realising that the state looked stranged, it wasn’t until I took a step back that I noticed the actual bug & to my amazement the bug was present in a couple of methods (I hate copy & paste). To be honest this to date is probably one of my biggest cock-ups & potentially one of the damaging bugs I’ve introduced into a system in a long time, I’d hate to see how many other system have fallen prey to this oversight.

Just so that there is an example out there for future reference here is the corrected version of the above code.

handle_call({disconnect, User}, _From, State) ->
{Reply,NewTree} =
case gb_trees:is_defined(User, State#foobarz.users) of
true ->
{{ok, "User dropped"},
gb_trees:delete(User, State#foobarz.users)};
false ->
{{error, "Unable to drop group."},
State#foobarz.users}
end,
{reply, Reply, State#foobarz{ users = NewTree }};

Spawning processes within processes

I was stuck on this for a few days and still think there is a better way I about this, spawning a new process is so simple it didnt occur to me that when creating a gen_server the gen_server was actually spawning a new process (duh, read the source). I subconciously took this in whilst reading up on it but when it came down to doing it myself it had me stumped, here is what I was trying to do:

spawn_link(mod_gen_server,start_link,arg).

The process I was trying to spawn was a gen_server and already spawning a process within itself, so what I was effectively trying do was spawn_link something that was trying to spawn a another seperate process, not quite what I wanted & Erlang definately didn’t like it, all I needed was to call the process and to link/monitor it. That simple.

Multiple processes

Well this was another one, once I worked out how to link a process I soon realised I needed that child process to have a unique module name, this is another one that may seem hard but in practicality it is yet again very simple.

start(Name,Desc) ->
    gen_server:start_link({global, Name}, ?MODULE, start, [Name,Desc]).

This way, the process is given the name passed the user, there are probably safer secure ways of doing this but for what I need at the moment, it works at treat.

Module definition not the same as file name

Is such a basic thing to oversee especially coming from a language where file names aren’t so important, its a simple one to fix but yet a common gotcha. Emacs has a nice lil feature that which basics alerts you to this fact and asks you if you want to changed it.

Creating result structures

Yet again another simple practice but if you come from a purely OO/C based language environment & just finished working on a OO based project or something, watch out for this, getting used to it takes some time. Simple rule:

  • Manipulate Term, giving unique names to needed terms
  • Store resulting terms into new pattern

Lists can simply be manipulated by postfixing a new list element to an existing on:

{{FName,_MName,LName},Address,Number,_OtherStuff} = UserDetails,
[[FName,SName,Address,Number]|Results].

Parsing XML

Whilst messing around with my Twitterl project, I quickly realised one thing, how was I to parse XML (second nature to me in PHP/C++) in Erlang though it did take me a while to get my head around, thanks to some related code by Nick Gerakines, reading his code definately shed some light on the area.

This is still a doozy, hopefully I’ll find a better way of doing this, basically we use xmerl_xpath:string/2 to retrieve an XML element, this result is a tuple, so we need to work on it further to retrieve the actual value, this could simply be done by:

[#xmlText{value = Value}] = xmerl_xpath:string(XML, "/item/title/text()").

I had implemented a variation of this code within my module but found it slow, so after reading a comment from Nick Gerakine’s in relation to my project, I had a look at his code for his Twitter API & noticed a bit of code that did they type of functionality I wanted, I’d like to find a way to improve this some more, mainly by looping over a record (still to learn) but it definately does the trick & imspires me to write more elegant code:

parse_item(Node) ->
    Item = #tweet {
      title = format_text(Node, ["/item/title/text()"],""),
      pubDate = format_text(Node, ["/item/pubDate/text()"],""),
      link = format_text(Node, ["/item/link/text()"], "")
    },
    Item.

This method also takes two Terms, the XML node & a list of XPath strings, it passes the node off to format_text, which does most of the work:

format_text(_, [], Result) -> Result;
format_text(Xml, [Xpath | Tail], Result) ->
    Results = lists:foldr(
        fun(#xmlText{value = Value}, Acc) ->
            lists:append(Value, Acc);
        (_, Acc) -> Acc
        end,
        Result,
        xmerl_xpath:string(Xpath, Xml)
    ),
    format_text(Xml, Tail, Results).

Here we basically loop over the XPath list retrieving the results and storing them in a list called Results, when the list of XPaths is empty we return the result, as most of the time I only need the one XPath it this method wasn’t introduced until i needed to play around with Twitters Users & Statuses XML.

Well that generally covers it, there is still a few more things a I want to cover & no doubt I’ll add more to this post as

Stuff I have yet to learn

  • Still not got round to getting getting auto-complete work under emacs.
  • Error and log handling principles, how to recover from a termination.
  • erl_crash.dump, is totally alien to me.
  • RESTful applications (have done some R&D but have yet to implement the actual functionality).
  • TDD & the whole process is still not well documented, ideally I want to be able to develop using the same TDD process I use in OOP.
  • Find out how to inplement a continous integration system to make development smoother.

Admittedly alot of this is more best practices and handling production code but I’m never happy until I have these things ironed out ;).

Well its been near four months since I started coding Erlang & I’ve got to say I’m enjoying the hell out of it, I thought the curve would be steep & admittedly initially it was, I still have moments of total confusion but I’m sure the more I play with Erlang the less this will happen, hell I still have those moments in C & PHP so nothing really different there.

Next

The next post will focus on how to put all this together to make an OTP system & how to main builds using Erlware’s Faxien & Sinan.

3 comments

  1. [...] focussing on TDD, PHP & programminng in general. « Lighting up the tunnErl Pt 1. Lighting up the tunnErl Pt. 3 – AftErl the basics » Lighting up the tunnErl Pt 2 – Discovering a new wErld January 5, 2009 This post [...]


  2. Could you talk about your productivity with Erlang vs C++ (or PHP)? Thanks!


    • Good point I have thought about comparing the productivity of Erlang vs C type language some what but don’t think its not particularly fair atm, seeing as I have been coding in C like languages for donkeys as opposed to the few months of Erlang but I will certainly do some code comparisons (Erlang vs PHP/C, etc ) in the near future and possibly in a few months time when I am a bit more than just proficient in Erlang I’ll attack this full on.

      I will say this tho, with my current project, though it took a couple of weeks (admittedly was working on other stuff also) work out how to implement the functionality, simply to implement the process handling and management of shutting down processes. In all honesty the functionality I have implemented could quite easily of been thrown together by a seasoned Erlanger is an hour max.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: