Monthly Archives: March 2015

Ruby Tuesday – Array#delete

Today’s Ruby Tuesday covers Array#delete.

Array#delete removes all items in the array that are equal to the object passed as an argument to the method.

list = [1, 2, 1, 3, 1, 5]
# => [1, 2, 1, 3, 1, 5]
list.delete(1)
# => 1
list
# => [2, 3, 5]
list.delete(3)
# => 3
list
# => [2, 5]

mississippi = "Mississippi".chars
# => ["M", "i", "s", "s", "i", "s", "s", "i", "p", "p", "i"]
mississippi.delete("s")
# => "s"
mississippi
# => ["M", "i", "i", "i", "p", "p", "i"]

Array#delete by default returns nil if the item was not found in the list.

no_fours = [1, 2, 3, 5, 6, 7]
# => [1, 2, 3, 5, 6, 7]
no_fours.delete(4)
# => nil

But if you decide to use Array#delete to remove nils from the array, you may not be able to check that it was removed because nil is returned as the value that was removed.

contains_nil = [1, "a", nil, :foo]
# => [1, "a", nil, :foo]
contains_nil.delete(nil)
# => nil
contains_nil
# => [1, "a", :foo]

If you come across that scenario, Array#delete allows you to specify a block that gets called if the item was not found in the array.

contains_nil2 = [1, "a", nil, :foo]
# => [1, "a", nil, :foo]
contains_nil2.delete(nil) {:bar}
# => nil
contains_nil2
# => [1, "a", :foo]
contains_nil2.delete(nil) {:bar}
# => :bar

As mentioned above, Array#delete removes all items that are equal to the argument passed in. The catch is Ruby has three different equality comparisons: ==, eql?, and equal?. So let’s try a couple of different scenarios to see which equality operator Ruby uses when doing an equality check.

mississippi = "Mississippi".chars
# => ["M", "i", "s", "s", "i", "s", "s", "i", "p", "p", "i"]
mississippi.delete("s")
# => "s"
"s" == "s"
# => true
"s".eql? "s"
# => true
"s".equal? "s"
# => false

So it looks like it is not equal?, but we still have == and eql? left that it could be, so let’s try another scenario.

integers = [1, 2, 3, 4]
# => [1, 2, 3, 4]
integers.delete(1.0)
# => 1
integers
# => [2, 3, 4]
1 == 1.0
# => true
1.eql? 1.0
# => false
1.equal? 1.0
# => false

As we can see here, 1 was removed when 1.0 was passed as an argument, so we see that Array#delete uses the == equality check to determine if the items are equal.

–Proctor

Erlang Thursday – erlang:list_to_atom/1

Today’s Erlang Thursday covers erlang:list_to_atom/1.

erlang:list_to_atom/1 takes a string, and returns an Erlang atom.

list_to_atom("foo").
% foo
list_to_atom("Foo").
% 'Foo'
list_to_atom("Foo_bar").
% 'Foo_bar'
list_to_atom("foo_bar").
% foo_bar
list_to_atom("foo"++"bar").
% foobar
list_to_atom("Erlang").
% 'Erlang'
list_to_atom("Elixir").
% 'Elixir'

This can be useful if you are having to create keys or identifiers based off strings read in from outside the system, such as parsing a CSV style header.

lists:map(fun erlang:list_to_atom/1,
          string:tokens("firstName,lastName,age,gender,preferredName,dateOfBirth", ",")).
% [firstName,lastName,age,gender,preferredName,dateOfBirth]

You do need to be careful when using erlang:list_to_atom/1 on strings acquired from the outside world of your program, as it only handles strings with character values under 256. But any character value1 under 256 is fair game to be turned into an atom.

list_to_atom("Joe, Mike, and Robert").
% 'Joe, Mike, and Robert'
list_to_atom("it's").
% 'it's'
list_to_atom("heyn").
% 'heyn'€
list_to_atom("with_supported_char"++[255]).
% with_supported_charÿ
list_to_atom("with_non_supported_char"++[256]).
% ** exception error: bad argument
%      in function  list_to_atom/1
%         called as list_to_atom([119,105,116,104,95,110,111,110,95,115,117,112,112,111,114,
%                                 116,101,100,95,99,104,97,114,256])

–Proctor


1. Remember that character values are non-negative integer values as well. 0-255 inclusive. back

Ruby Tuesday – String#to_sym

Today’s Ruby Tuesday covers String#to_sym.

String#to_sym operates on a String object, and turns the string into a Symbol.

Why would someone want to have a Symbol instead of a String?

The main reason is that Symbols are created once for the lifetime of a program, and use the same reference. Every reference to the symbol :foo refers to the same instance of :foo, and it doesn’t matter when, or where, in the program lifetime it is.

Strings on the other hand are unique objects, so two strings of the same value are not necessarily the same object, and depends on when, where, and how they were created.

"foo".to_sym
# => :foo
"Foo".to_sym
# => :Foo
"foo".__id__ == "foo".__id__
# => false
:foo.__id__ == :foo.__id__
# => true

One case where this might be useful is where you have to tokenize strings, and are going to want to use them as keys for a map. Think header columns in a CSV file, but please don’t write your own CSV parser, as Ruby has a nice one built in.

"firstName,lastName,preferred name,age".split(",").map(&:to_sym)
# => [:firstName, :lastName, :"preferred name", :age]

The other place where you might want to use String#to_sym is when you are trying to create Symbols that cannot easily be created by just using the normal symbol notation.

":foo".to_sym
# => :":foo"
"peanut butter".to_sym
# => :"peanut butter"

–Proctor

Erlang Thurday – ordsets:is_disjoint/2

Today’s Erlang Thurday covers ordsets:is_disjoint/2.

There are times in your in your coding day where you have problems where you need to know if given a list of items that none of those items appear in a secondary list.

Your first intuition might be to write out the code as described, like such:

not( lists:any(fun(Item) -> lists:member(Item, [1, 3, 5, 7]) end, [2, 4, 6])).
% true

And while that is accurate, if you redefine your problem in more mathematical terms, you can start to think in sets. When you start to think in terms of sets, you realize that you can check to see if the intersection of the two sets is the empty set.

ordsets:intersection([1, 3, 5, 7], [2, 4, 6]) =:= [].
% true

This is becoming not only more concise, but also more explicit about what you are trying to check.

We can do better still, by checking if the lists are disjoint sets. Enter ordsets:is_disjoint/2.

ordsets:is_disjoint/2 takes two lists, and returns true if no elements are in common.

ordsets:is_disjoint([1, 3, 5, 7], [2, 4, 6]).
% true
ordsets:is_disjoint([1, 2, 3, 5, 7], [2, 4, 6]).
% false

Because ordsets:is_disjoint/2 operates against two lists, we do not have to make sure the elements are unique prior to calling ordsets:disjoint/2.

ordsets:is_disjoint([1, 1, 3, 5, 7, 5, 3], [2, 4, 2, 2, 6]).
% true
€ordsets:is_disjoint([1, 2, 3, 5, 7], [2, 4, 2, 2, 6]).
% false

And if either list passed to ordsets:is_disjoint/2 is an empty list, the result is that the lists are disjoint.

ordsets:is_disjoint([1, 2, 3, 5, 7], []).
% true
ordsets:is_disjoint([], [2, 4, 6]).
% true
ordsets:is_disjoint([], []).
% true

And if you are curious, by running ordsets:is_disjoint/2 through timer:tc/3, we can see that as soon as Erlang knows that the sets are not disjoint, it returns false. And if you remember from the previous Erlang Thrusday on timer:tc/3, the return value is a tuple with the first element being the number of microseconds it took to complete.

timer:tc(ordsets, is_disjoint, [lists:seq(1, 1000000), lists:seq(2000000, 3000000)]).
% {19032,true}
€timer:tc(ordsets, is_disjoint, [lists:seq(1, 1000000), lists:seq(1, 3000000)]).
% {2,false}

–Proctor

Ruby Tuesday – Set#disjoint?

Today’s Ruby Tuesday is on Set#disjoint?.

To start using Sets, we first need to require Set.

require 'set'

Set#disjoint? is useful when you want to make sure a group of items is not in another group of items.

Without using Set#disjoint? you would probably be writing something like the following:

[1, 3, 1, 5, 7].none? { |item| [2, 4, 6, 8].include? item }
# => true

While this works, it does not make the intent of what you are trying to do explicit, which is where Set#disjoint? comes in.

Set#disjoint? operates against a set, and takes another Set as it’s argument, and checks to see if there is no common element between the two sets.

[1, 3, 1, 5, 7].to_set.disjoint? [2, 4, 6, 8].to_set
# => true
[1, 2, 3].to_set.disjoint? [1, 3, 5, 7].to_set
# => false

If a nil is present in both sets, Set#disjoint? treats the nils as equal and returns false.

[2, nil].to_set.disjoint? [1, 3, 5, 7, nil].to_set
=> false

Set#disjoint? works against empty sets as well.

[].to_set.disjoint? [1, 2, 3].to_set
# => true
[1, 2, 3].to_set.disjoint? [].to_set
# => true

If two empty sets are passed to Set#disjoint?, it returns true, as both sets have no elements, and therefore no elements in common.

[].to_set.disjoint? [].to_set
# => true

–Proctor

Erlang Thursday – lists:flatmap/2

Today’s Erlang Thursday is on lists:flatmap/2.

The selection of this function comes from some Elixir exercises I was doing earlier in the week, and was getting confused because when using the Enum.flat_map/2 function in Elixir wasn’t working as I expected it to.

I circled back to Erlang to check the behavior of lists:flatmap/2 in Erlang, and after getting over my confusion that the flatmap function wasn’t working in Erlang, the common behavior made sense, and I realized I contorted the concept in my head and it was time to take a step back and figure out what lists:flatmap/2 actually does.

Somehow what I had convinced myself that lists:flatmap/2 did was to take a list of items that were nested arbitrarily deep, and mapped over the flattened list in the equivalent of this:

lists:map(fun(X) -> X * X end, lists:flatten([1, [[2, [3]], 4]])).
% [1,4,9,16]

Looking closer at the documentation, after even trying Ruby’s flat_map function, it was clear I had completely confused myself as to how it should work, so time to carefully read the Erlang docs.

They stated that lists:flatmap/2 takes a function that takes an item of type A and returns a list of items that are of type B, and that the second argument to lists:flatmap/2 was a list of items of type A.

Being already confused based off how I thought it should work, it wasn’t registering until I saw the part of the documentation that describes that lists:flatmap/2 behaves as if defined as

flatmap(Fun, List1) ->
    append(map(Fun, List1)).

This started to click for me for what lists:flatmap/2 was actually doing. In the picture that I had in my head, I was expecting to flatten the list first, and then map over the items, but it does the map first, and then does only the flatten, but of only one level deep.

lists:flatmap(fun({Item, Count}) -> 
                  lists:duplicate(Count, Item)
              end,
              [{a, 1}, {b, 2}, {'C', 3}, {'_d_', 4}]).
% [a,b,b,'C','C','C','_d_','_d_','_d_','_d_']

And if we pass those values to the “equivalent” behavior of calling map and then calling append on the list returned from map, we see the results are the same.

lists:append(
    lists:map(fun({Item, Count}) ->
                  lists:duplicate(Count, Item)
              end,
              [{a, 1}, {b, 2}, {'C', 3}, {'_d_', 4}])).
% [a,b,b,'C','C','C','_d_','_d_','_d_','_d_']

And to further clarify, lists:flatmap/2 doesn’t even do a flatten on the resulting list, but simply adjoins the lists that were returned from the mapping function. This can be seen below, as we can see there is still an nested list structure in the results, and the resulting list is not only one level deep.

lists:flatmap(fun(X) -> [X, [X]] end, [a, b, c, d]).
% [a,[a],b,[b],c,[c][/c],d,[d]]

Hope this can save you the confusion I managed to get myself into,
–Proctor

Ruby Tuesday – Enumerable#flat_map

Today’s Ruby Tuesday is on Enumerable#flat_map.

Enumerable#flat_map became this week’s topic, because yesterday I was doing some exercises in Elixir, and was thinking I could use the flat_map that is defined in Elixir to solve the solution, but it wasn’t working the way I thought it should.

I was getting confused with why it was failing, and I went to check my assumptions of how flatmap works in Erlang, and it behaved the same way. Which, after seeing it, is unsurprising given their common runtime, but I couldn’t figure out where my mental model of flat_map was coming from, other than the name alone.

It was time to pull up a Ruby REPL, and see if Ruby was behaving the way I thought it “should” be behaving.

[1, [[2], 3]].flat_map {|x| x * x}
# TypeError: no implicit conversion of Array into Integer
# from (pry):11:in `*'

No! It failed in Ruby as well.

I had myself convinced that flat_map was the equivalent of doing a Array#flatten, and then calling Enumerable#map on the result.

[1, [[2], 3]].flatten.map {|x| x * x}
# [1, 4, 9]

This is the equivalent of what I was trying to get, and flat_map wasn’t working for me, and obviously, it is not the way Enumerable#flat_map behaves either.

After looking deeper, I realized that somehow, I had the application of the conceptual flatten and map methods swapped.

The way Enumerable#flat_map actually behaves is as if you called map on the Enumerable, and then called flatten on the results.

[[:a, 1], [:b, 2], [:c, 3]].flat_map {|item, count| Array.new(count, item) }
# [:a, :b, :b, :c, :c, :c]

[[:a, 1], [:b, 2], [:c, 3]].map {|item, count| Array.new(count, item) }
# [[:a], [:b, :b], [:c, :c, :c]]
[[:a, 1], [:b, 2], [:c, 3]].map {|item, count| Array.new(count, item) }.flatten
# [:a, :b, :b, :c, :c, :c]

And if an Enumerable is not returned by the map block passed to Enumerable#flat_map, it behaves the same as if you just called Enumerable#map.

[1, 2, 3].flat_map {|x| x * x}
# [1, 4, 9]
[1, 2, 3].map {|x| x * x}
# [1, 4, 9]

Here is hoping that this saves you some confusion as well.
–Proctor

Erlang Thursday – calendar:local_time_to_universal_time_dst/1

In honor of the time change this weekend, today’s Erlang Thursday is on calendar:local_time_to_universal_time_dst/1.

To better understand what calendar:local_time_to_universal_time_dst/1 is doing, it will be contrasted to calendar:local_time_to_universal_time/1.

This coming Sunday is we move the clock forward an hour at 2 AM, so let us see what the time is in UTC right at 1:59 AM.

calendar:local_time_to_universal_time({{2015,3,8},{1,59,59}}).
% {{2015,3,8},{7,59,59}}

Now let’s see what calendar:local_time_to_universal_time/1 returns for 2 AM, which we never actually hit, since the time changes straight to 3 AM.

calendar:local_time_to_universal_time({{2015,3,8},{2,00,00}}).
% {{2015,3,8},{8,0,0}}

But as we dig into the library we see that there is also calendar:local_time_to_universal_time_dst/1. calendar:local_time_to_universal_time_dst/1 returns a list of time tuples, and we start to see why this is important.

This time, let’s call calendar:local_time_to_universal_time_dst/1 with the 2 AM hour and see what is returned.

calendar:local_time_to_universal_time_dst({{2015,3,8},{2,00,00}}).
% []

An empty list!

This makes sense if you take a moment to think about it, as 2 AM doesn’t ever happen so there is no UTC time that it would map to.

And if we look at what happens when we make the jump to 3 AM, we get 8 AM UTC, which is just a second later in UTC, then the time in UTC at 1:59:59 AM.

calendar:local_time_to_universal_time_dst({{2015,3,8},{3,00,00}}).
% [{{2015,3,8},{8,0,0}}]

That covers the case for this coming weekend, so let is check out what is going to happen later this year on November 1st when the clocks move back an hour.

To start with a baseline, let’s see what time it is in UTC at 12:59:59 AM, as we know we only encounter that time once.

calendar:local_time_to_universal_time({{2015,11,1},{0,59,59}}).
% {{2015,11,1},{5,59,59}}
calendar:local_time_to_universal_time_dst({{2015,11,1},{0,59,59}}).
% [{{2015,11,1},{5,59,59}}]

And even though the time change happens a 2 AM, we still only encounter that once, as we move straight to 1 AM the first time.

calendar:local_time_to_universal_time_dst({{2015,11,1},{2,0,0}}).
% [{{2015,11,1},{8,0,0}}]

So at 12:59:59 AM, we are at 5:59:59 AM UTC, and at 2 AM we are at 8 AM UTC. Now comes the tricky part, 1 AM.

1 AM is an odd case, as we will live through 1 AM twice that night. So let’s see what Erlang does with that.

First, we see what time it is in UTC when we call calendar:local_time_to_universal_time with 1 AM.

calendar:local_time_to_universal_time({{2015,11,1},{1,00,00}}).
% {{2015,11,1},{6,0,0}}

We get 6 AM UTC. We move from 5:59:59 AM UTC to 6 AM UTC; and that makes sense, until we start to wonder about the second time we encounter 1 AM.

Now, let’s see what time it is in UTC when we call calendar:local_time_to_universal_time_dst/1 with 1 AM.

calendar:local_time_to_universal_time_dst({{2015,11,1},{1,0,0}}).
% [{{2015,11,1},{6,0,0}},{{2015,11,1},{7,0,0}}]

We get a list with two times in UTC, one for 6 AM UTC when we encounter 1 AM the first time, and one for 7 AM UTC for when we encounter 1 AM the second time around. This happens for every time in the 1 AM hour as well.

calendar:local_time_to_universal_time_dst({{2015,11,1},{1,59,59}}).
% [{{2015,11,1},{6,59,59}},{{2015,11,1},{7,59,59}}]

By returning a list of either no items, one item, or two items, Erlang gives you the most accurate information for you to use, and allow for you to make the informed decision for how you are expecting to handle the times, instead of making some of the decisions for you even if they don’t align with your system’s view of the world.

–Proctor

Ruby Tuesday – Time#utc

In honor of the time change this weekend, today’s Ruby Tuesday is on Time#utc.

In the past few years I have had to deal with numerous bugs crop up around the changes between Daylight time and Standard time. In honor of all the time conversion issues I have dealt with in the past couple of years, I thought it would be timely, pun absolutely intended, to explore time conversions. As as this deals with local time vs UTC time conversions, let me preface this with a warning:

Here there be dragons.

With that said, let us see if we can channel the spirit of Dirk the Daring, and attempt to face down some of those dragons.

For this upcoming time change we will be moving the clocks forward an hour at 2 AM.

Let’s see what time it is right before the time change.

Time.new(2015,3,8, 1,59,59).utc
# 2015-03-08 07:59:59 UTC

And when 2 AM “arrives”, we move the clock forward and it becomes 3 AM.

Time.new(2015,3,8, 3,00,00).utc
# 2015-03-08 08:00:00 UTC

So far, all that looks good. But what happens at 2 AM?

Time.new(2015,3,8, 2,00,00).utc
# 2015-03-08 08:00:00 UTC

Though 2 AM on March 8th of 2015 never really happens, at least here in the Dallas/Fort Worth area, we do get a valid time returned when we call Time#utc, and the result is the same as when we called the utc method for 3AM.

That was interesting; as far if that is good or bad, that depends on how well your system can handle that behavior.

Now onto what happens later in the year when we set our clocks back an hour.

The first time that 2 AM rolls around we set our clocks back an hour to 1 AM, so let’s see what time 12:59:59 AM is in UTC, since we know that only happens once.

Time.new(2015,11,1, 00,59,59).utc
# 2015-11-01 05:59:59 UTC

And we only hit 2 AM only once as well, since the first time we came to 2 AM we all skipped it and relived the 1 AM hour.

Time.new(2015,11,1, 2,00,00).utc
# 2015-11-01 08:00:00 UTC

So now the question is: “What time is it at 1 AM?”

Time.new(2015,11,1, 1,00,00).utc
# 2015-11-01 07:00:00 UTC

Hrmm…..

It looks like we only get the time for 1 AM for the second time it rolls around, as it returns 7 AM UTC, and we have completely lost the 6 AM UTC hour. Dragons!

Hopefully you can avoid most of these issues and just work in UTC or epoch seconds across your systems. But as our systems continue to interact with more and more outside systems, the likelihood of having to deal with these issues increase, so please be aware.

–Proctor