Erlang Thursday – lists:zip/2

Today’s Erlang Thursday is lists:zip/2.

lists:zip/2 returns a new list of two-tuples, from the corresponding elements of the two lists passed as arguments.

lists:zip([a, b, c, d], [1, 2, 3, 4]).
% [{a,1},{b,2},{c,3},{d,4}]
lists:zip([a, b, c, d], [{1, 3}, {2, 5}, {3, 7}, {4, 11}]).
% [{a,{1,3}},{b,{2,5}},{c,{3,7}},{d,{4,11}}]
lists:zip([a, b], [fun lists:map/3, fun lists:foldl/3]).
% [{a,#Fun<lists.map.3>},{b,#Fun<lists.foldl.3>}]

If the lists are not of the same length a function clause match exception is thrown.

lists:zip([a, b, c, d], [1, 2, 3]).                        
% ** exception error: no function clause matching lists:zip([d],[]) (lists.erl, line 385)
%      in function  lists:zip/2 (lists.erl, line 385)
%      in call from lists:zip/2 (lists.erl, line 385)
lists:zip([a, b, c], [1, 2, 3, 4]).                        
% ** exception error: no function clause matching lists:zip([],[4]) (lists.erl, line 385)
%      in function  lists:zip/2 (lists.erl, line 385)
%      in call from lists:zip/2 (lists.erl, line 385)

There is also a 3-arity version of the zip function lists:zip3/3, which behaves like lists:zip/2 but takes three lists as arguments instead of two.

lists:zip3([a, b, c, d], [1, 2, 3, 4], ["alpha", "bravo", "charlie", "delta"]).         
% [{a,1,"alpha"},{b,2,"bravo"},{c,3,"charlie"},{d,4,"delta"}]

If you need to combine the arguments in a different manner, you can use lists:zipwith/3, or lists:zipwith3/4, each of which takes as the first argument a 2-arity combining function.

lists:zipwith(fun(X, Y) -> X * Y end, [1, 2, 3, 4], [2, 3, 4, 5]).
% [2,6,12,20]
lists:zipwith(fun(X, Y) -> X + Y end, [1, 2, 3, 4], [2, 3, 4, 5]).                                                
% [3,5,7,9]
lists:zipwith3(fun(X, Y, Z) -> X * Y * Z end, [1, 2, 3, 4], [2, 3, 4, 5], [4, 3, 2, 1]).
% [8,18,24,20]
lists:zipwith3(fun(X, Y, Z) -> {{X, Y}, Z} end, [a, b, c, d], [1, 2, 3, 4], ["alpha", "bravo", "charlie", "delta"]).
% [{{a,1},"alpha"},
%  {{b,2},"bravo"},
%  {{c,3},"charlie"},
%  {{d,4},"delta"}]

–Proctor

,

Leave a Comment

Ruby Tuesday – Enumerable#zip

Today’s Ruby Tuesday covers Enumerable#zip.

Enumerable#zip weaves together elements of multiple Enumerables. It iterates through each item in the enum and combines it with the corresponding element from each of the arguments passed to zip, resulting in an array of arrays.

[:a, :b, :c].zip([1, 2, 3])
# => [[:a, 1], [:b, 2], [:c, 3]]
[:a, :b, :c].zip([1, 2, 3], [:foo, :bar, :baz])
# => [[:a, 1, :foo], [:b, 2, :bar], [:c, 3, :baz]]

If an item in the argument list is shorter then the enum that zip is being called on, nil is used where the item would have been added.

[:a, :b, :c].zip([1], [:foo, :bar, :baz, :snafu])
# => [[:a, 1, :foo], [:b, nil, :bar], [:c, nil, :baz]]

The length of the returned array is the length of the enum that zip is being called on. Any extra elements in the enumerable passed to zip are ignored, and if are in a lazy enumerble not computed.

[:a, :b, :c].zip([1, 2, 3, 4], [:foo, :bar, :baz, :snafu])
# => [[:a, 1, :foo], [:b, 2, :bar], [:c, 3, :baz]]

### Only takes what is needed ###
require 'benchmark'
# => true
Benchmark.measure do
  [:a, :b].zip( (1..1_000_000_000_000).lazy.map{|x| x*x*x} ) 
end
# => #<Benchmark::Tms:0x007fd5749d9288
#  @cstime=0.0,
#  @cutime=0.0,
#  @label="",
#  @real=4.3e-05,
#  @stime=0.0,
#  @total=0.0,
#  @utime=0.0>

#### For much much less, took 8+ seconds ####
Benchmark.measure do
  [:a, :b].zip( (1..10_000_000).map{|x| x*x*x} )   
end
# => #<Benchmark::Tms:0x007faed4779e00
#  @cstime=0.0,
#  @cutime=0.0,
#  @label="",
#  @real=8.616259,
#  @stime=0.7200000000000002,
#  @total=8.53,
#  @utime=7.809999999999999>

Enumerable#zip can also be given a block. If a block is provided, the combined array of items is passed to the block, and not returned as a result. This behavior is distinct from Enumerable#each in that each still returns the Enumerable it was invoked on, where as zip returns nil as it’s result when a block is given.

[:a, :b, :c].zip([1, 2, 3, 4], [:foo, :bar, :baz, :snafu]) { |items| puts items.to_s }
# [:a, 1, :foo]
# [:b, 2, :bar]
# [:c, 3, :baz]
# => nil
[:a, :b, :c].zip([1, 2, 3, 4], [:foo, :bar, :baz, :snafu]).each { |items| puts items.to_s }
# [:a, 1, :foo]
# [:b, 2, :bar]
# [:c, 3, :baz]
# => [[:a, 1, :foo], [:b, 2, :bar], [:c, 3, :baz]]

–Proctor

,

Leave a Comment

Erlang Thursday – lists:foldl/3 and lists:foldr/3

Today’s Erlang Thursday is lists:foldl/3 and lists:foldr/3.

lists:foldl/3 is Erlang’s version of the reduce function. lists:foldl/3 takes a function, an initial accumulator value, and a list, and returns a single value.

The first argument given to foldl is a function that takes two arguments, the item currently iterating over, and the accumulated value. The result of applying this function is used as new new value for the accumulator for the following item, or if no items are left it becomes the result of foldl.

The next argument to lists:foldl/3 is an initial value of the accumulator. This is different than some other languages, as those languages will take the initial value for the accumulator as an optional value and use the first item to be traversed as the default value for the accumulator. But in Erlang an initial value for the accumulator is a required argument to both lists:foldl/3 and lists:foldr/3.

The third, and last, argument to foldl is the list to iterate over.

lists:foldl(fun(X, Sum) -> Sum + X end, 0, [1, 2, 3, 4, 5]).
% 15
lists:foldl(fun(X, Product) -> Product * X end, 1, [1, 2, 3, 4, 5]).
% 120
lists:foldl(fun(X, Accum) -> io:format("~p ", [X]) end, void, [1, 2, 3, 4, 5]).
% 1 2 3 4 5 ok
lists:foldl(fun(X, Accum) -> io:format("~p ", [X]), Accum end, void, [1, 2, 3, 4, 5]).
% 1 2 3 4 5 void
lists:foldl(fun(X, Accum) -> Accum + X end, 1, []).               
% 1
lists:foldl(fun(X, Result) -> lists:umerge(Result, X) end, [], [[1, 2, 3], [3, 5, 8], [11, 13, 17]]).
% [1,2,3,5,8,11,13,17]

The Erlang lists module also contains the function foldr/3 which traverses the list from left to right, or from the last item in the list to the first.

lists:foldr(fun(X, Accum) -> io:format("~p ", [X]), Accum end, void, [1, 2, 3, 4, 5]).
% 5 4 3 2 1 void
lists:foldr(fun(X, Accum) -> Accum + X end, 1, []).
% 1

The Erlang documentation points out that lists:foldl/3 is generally preferable over lists:foldr/3 because lists:foldl/3 is tail recursive, where lists:foldr/3 is not.

,

Leave a Comment

cURL and Squid

Yesterday I was attempting to use curl to import data into our development servers to load up lots of data for a demo, and my curl command kept spitting back an error.

HTTP/1.0 417 Expectation Failed
Server: squid/3.1.12
Mime-Version: 1.0
Date: Wed, 10 Dec 2014 13:50:35 GMT
Content-Type: text/html
Content-Length: 3688
X-Squid-Error: ERR_INVALID_REQ 0
Vary: Accept-Language
Content-Language: en
X-Cache: MISS from localhost
Via: 1.0 localhost (squid/3.1.12)
Connection: close
....
<p>Some possible problems are:</p>
<ul>
<li><p>Missing or unknown request method.</p></li>
<li><p>Missing URL.</p></li>
<li><p>Missing HTTP Identifier (HTTP/1.0).</p></li>
<li><p>Request is too large.</p></li>
<li><p>Content-Length missing for POST or PUT requests.</p></li>
<li><p>Illegal character in hostname; underscores are not allowed.</p></li>
<li><p>HTTP/1.1 <q>Expect:</q> feature is being asked from an HTTP/1.0 software.</p></li>
</ul>
....
<div id="footer">
<p>Generated Wed, 10 Dec 2014 13:50:35 GMT by localhost (squid/3.1.12)</p>
<!-- ERR_INVALID_REQ -->
</div>
</body></html>

Sadly, I wasted time by not thoroughly reading the error message, as I was attempting to go too fast with the goal to get extra data loaded for the demo. Finally, after taking a look at it this morning with fresh eyes, I saw the error was coming from Squid, and was not a response from our development servers complaining about some misconfigured settings or bad data format.

Finding it was had to do with Squid on my local machine, I was able to find that Squid (v3.1) was returning a 417 status code when using cURL’s default header settings.

The fix when using v3.1 of Squid is to add the option -H “Expect:” to the curl command to take off the Expect header.

The better option, if you can manage it, is to upgrade past v3.1 of Squid to at least v3.2, which claims HTTP/1.1 support. More information about the HTTP/1.1 support of Squid can be found here: http://wiki.squid-cache.org/Features/HTTP11

—Proctor

,

Leave a Comment

Ruby Tuesday – Enumerable#reduce aka inject

Today’s Ruby Tuesday is Enumerable#reduce, also known as Enumerable#inject.

Enumerable#reduce works against an enum and takes a initial value for the accumulator and two parameter block. The first parameter to the block is the current accumulated value, and second parameter is the current element in the iteration. If no accumulator value is specified, the first item in the enum is taken to be used as the accumulator value.

["baz","Supercalifragilisticexpialidocious", "qwerty"].reduce("") do |shortest, item|
  item.length < shortest.length ? item : shortest
end
# => ""
["baz","Supercalifragilisticexpialidocious", "qwerty"].reduce do |longest, item|
  item.length > longest.length ? item : longest
end
# => "Supercalifragilisticexpialidocious"
["baz", "Supercalifragilisticexpialidocious", "qwerty"].reduce do |shortest, item|
  item.length < shortest.length ? item : shortest
end
# => "baz"

Calling reduce on an empty enum just returns the accumulator, or if no initial value for the accumulator given returns the result of calling first on the enum, which in the case of Array is nil.

[].reduce(:accum){|accum, item| accum * item}
# => :accum
[].reduce{|accum, item| accum * item}
# => nil

Enumerable#reduce can also take a symbol instead of a block, which is the name of the method to invoke on the accumulator value. This form of reduce also takes an initial accumulator value, or uses the first item as the accumulator.

[1, 2, 3, 4, 5].reduce(:+)
# => 15
[1, 2, 3, 4, 5].reduce(:*)
# => 120
[1, 2, 3, 4, 5].reduce(10, :*)
# => 1200
[100, 10, 2].reduce(:/)
# => 5
[100, 10, 2].reduce(1_000_000, :/)
# => 500

–Proctor

,

Leave a Comment

Erlang Thursday – lists:map/2

Today’s Erlang Thursday features lists:map/2.

lists:map/2 takes two arguments, a “mapping” function which takes a single argument, and a list of Erlang terms. The result is a new list of items that is the result of mapping the function applied to each value in the original list.

lists:map(fun(X) -> X + 1 end, [1, 2, 3, 4]).
% [2,3,4,5]
lists:map(fun(X) -> X * X end, [1, 2, 3, 4]).
% [1,4,9,16]

Because strings in Erlang are just lists of integers, you can map against strings as well.

lists:map(fun(X) -> X - 1 end, "IBM").
% "HAL"

On Functions in Erlang

If you look at the first example above, you see that the first argument we are passing is fun(X) -> X + 1 end. This is Erlang’s syntax for an anonymous function. An anonymous function takes the form:

fun(Args1) OptionalGuardClause1 ->
        Expression1, Expression2;
   (Args2) OptionalGuardClause2 ->
        Expression3, Expression4;
   (Args3) OptionalGuardClause3 ->
        Expression5, Expression6;
end

Because we have the power of normal functions, except for being able to recursively call an anonymous function before version 17.0, we can use anonymous functions with multiple clauses when passing to map.

lists:map(fun(X) when is_atom(X) -> atom; (X) -> nil end, [1, x, {x}, [], 'B']).
[nil,atom,nil,nil,atom]

Passing Existing Named Functions to lists:map/1

While in some cases an inline anonymous function may work, many times we would want to have a named function for clarity. We can pass named functions to map by using the qualified name of the function – module:function_name/arity – prefaced with fun. The examples below use math:log10/1 and erlang:is_atom/1 to demonstrate.

lists:map(fun math:log10/1, [1, 10, 100, 1000, 10000]).
[0.0,1.0,2.0,3.0,4.0]
lists:map(fun erlang:is_atom/1, [1, x, {x}, [], 'B']).
[false,true,false,false,true]

–Proctor

,

Leave a Comment

Ruby Tuesday – Enumerable#map aka collect

Today’s Ruby Tuesday method is Enumerable#map, also known as Enumerable#collect.

Enumerable#map is a function with a long lineage. Enumerable#map takes a set of items, and maps the inputs onto a new set of values via a transformation function, or in the case of Ruby, a block.

[1, 2, 3, 4].map{ |x| x + 1 }
# => [2, 3, 4, 5]
[1, 2, 3, 4].map{ |x| x * x }
# => [1, 4, 9, 16]

In Ruby, the map method is an alias of Enumerable#collect.

[1, 2, 3, 4].map{ |x| x * x }
# => [1, 4, 9, 16]
[1, 2, 3, 4].collect{ |x| x * x }
# => [1, 4, 9, 16]

Ruby also has some ways of taking existing methods, and passing them as the block to be invoked by the map function. If we want to do a transform by calling a method on the object passed to the block, we can prepend the name of the method with a &: and pass that to the map function as an argument.

['i', 'b', 'm'].map{ |x| x.upcase }
=> ["I", "B", "M"]
['i', 'b', 'm'].map(&:upcase)
=> ["I", "B", "M"]

You want to be careful with this though, as not all functions on an object are transformation functions but may modify the underlying object, which may cause confusion to readers of your code if they are expecting the original enumeration to be unchanged.

items = ['i', 'b', 'm']
# => ["i", "b", "m"]
items.map(&:upcase)
# => ["I", "B", "M"]
items
# => ["i", "b", "m"]
items = ['i', 'b', 'm']
# => ["i", "b", "m"]
items.map(&:upcase!)
#=> ["I", "B", "M"]
items
# => ["I", "B", "M"]

,

Leave a Comment

Erlang Thursday – lists:flatten/1

Today’s Erlang Thursday function is lists:flatten/1.

lists:flatten/1 flattens out an arbitrarily deep list of Erlang terms, into a “flattened” list.

lists:flatten([]).
% []
lists:flatten([a, b, c]).
% [a,b,c]
lists:flatten([a, b, [1, [x, y], 3], c]).
% [a,b,1,x,y,3,c]
lists:flatten([a, b, [1, [x, {some, tuple}], 3], c]).    
% [a,b,1,x,{some,tuple},3,c]

Be warned though, it flattens out all lists, as seen here

lists:flatten([a, "foo", b]).    
% [a,102,111,111,b]

You get the above lists with numbers in it, because under the covers, a string is just a list of integers, so you get the ASCII character codes for the letters f and o in "foo".

If you want the string to “remain”, you need to use the string as a binary type like this:

lists:flatten([a, <<"foo">>, b]).                       
% [a,<<"foo">>,b]

And as a bonus, there is also a lists:flatten/2, that takes a list to flatten, and another argument tail, which is the value to append to the newly flattened list.

lists:flatten([a, [1, [b, [2]]]], [x, y, z]).
% [a,1,b,2,x,y,z]

–Proctor

,

Leave a Comment

Ruby Tuesday – Array#flatten

Today’s Ruby Tuesday is Array#flatten.

Array#flatten returns a new array that has been recursively flattened into a one dimensional array.

a = [1, [:a, :b, :c], 3]
# [1, [:a, :b, :c], 3]
b = a.flatten
# [1, :a, :b, :c, 3]
b
# [1, :a, :b, :c, 3]
a
# [1, [:a, :b, :c], 3]

c = [1, [2, [3, 4, [5 ,6]]]]
# [1, [2, [3, 4, [5, 6]]]]
d = c.flatten
# [1, 2, 3, 4, 5, 6]
c
# [1, [2, [3, 4, [5, 6]]]]
d
# [1, 2, 3, 4, 5, 6]

[1, 2, 3].flatten
# [1, 2, 3]

Ruby also provides a method #flatten! which mutates the state of the object if you happen to require that behavior. And note, that calling flatten! on an array will return nil if the array was not modified.

c2 = [1, [2, [3, 4, [5 ,6]]]]
# [1, [2, [3, 4, [5, 6]]]]
c2.flatten!
# [1, 2, 3, 4, 5, 6]
c2
# [1, 2, 3, 4, 5, 6]

[1, 2, 3].flatten!
# nil

–Proctor

,

Leave a Comment

Erlang Thursday – lists:max/1

Today’s Erlang function is lists:max/1.

lists:max/1 takes one argument, a list of at least one term, and returns the item that has the greatest value when compared to the other items in the list. The list can be composed of any Erlang term:

lists:max([1, 39, 7, 63, 27, 52, 16]).
% 63
lists:max([q, w, a, r, c, f, m, b]).      
% w
lists:max([[1, 2, 3], [1, 2, 4]]).
% [1,2,4]

Erlang has a distinct order between the different types of terms,

number < atom < reference < fun < port < pid < tuple < list < bit string

allowing for the list passed to lists:max/1 to be comprised of disparate types.

lists:max([1, 2, 3, 4, a]).         
% a
lists:max([1, a, [foo, bar], {baz}]).
% [foo,bar]

And because a string in Erlang is just a list of numbers under the covers, we can even compare strings.

lists:max(["foo", "bar", "baz", "snafu"]).
"snafu" 

If you pass an empty list to lists:max/1 a no function clause matching error will be raised, since it expects the list to be at least composed of one term.

lists:max([]).
** exception error: no function clause matching lists:max([]) (lists.erl, line 326)

What are some of your favorite Erlang functions, or even just ones you would like to see a future Erlang Thursday post about?

And don't forget to check out the last Tuesday's Ruby Tuesday on Enumerable#max, if you would like a comparison between Erlang's max function and Ruby's max method.

--Proctor

,

Leave a Comment