Monthly Archives: September 2015

Ruby Tuesday – SecureRandom::uuid

Today’s Ruby Tuesday is on SecureRandom::uuid.

Sometimes you just want a unique identifier, and the more guaranteed to be unique the identifier is, the better.

Enter SecureRandom::uuid.

SecureRandom::uuid generates a new 128 bit identifier, and according to the RFC for a UUID, a.k.a GUID, is “guaranteed to be unique across space and time”.

While the math and avanced physics to prove that guarantee is beyond me, I have found that they are unqiue enough for every case that I have encountered so far. If you find otherwise I would love for you to let me know how you “broke” that guarantee.

To use SecureRandom::uuid, we first have to require 'securerandom'. And yes, this is one of those Ruby requires that does not have the name snake cased, but all one word.

require 'securerandom'
# => true
SecureRandom.uuid
# => "fc804c3a-def2-4739-a723-605992f055cc"
SecureRandom.uuid
# => "67986323-afe9-41c9-81d8-c91d13593534"
SecureRandom.uuid
# => "2deabd2c-55a1-48b5-8b6e-6ee96d56036e"
SecureRandom.uuid.class
# => String

This returns a Version 4 UUID, as can be seen by the 4 following the second dash in the UUID.

SecureRandom.uuid.class
# => String

Be warned though, that this doesn’t generate an actual UUID object, but just the String representation of a UUID, so if you need access to the raw bits of the UUID generated, you will likely need to track down another gem for that use case.

–Proctor

Erlang Thursday – erl_tar:extract/1

Today’s Erlang Thursday cover’s erl_tar:extract/1.

erl_tar:extract/1 takes a file, either as a binary tuple, file descriptor tuple, or filename, and extracts the contents of the tar out to the current directory.

Since we will need to have a tar file to extract, let’s create some files and add them to a new tar file.

$ echo "woof" > dog.txt
$ echo "meow" > cat.txt
$ echo "sparkle" > pony.txt
$ echo 'Wocka Wocka Wocka!' > bear.txt
$ tar -cvf animal_sounds.tar dog.txt cat.txt pony.txt bear.txt
a dog.txt
a cat.txt
a pony.txt
a bear.txt

And while we are at it, lets create a compressed version as well.

$ tar -cvzf animal_sounds.tar.gz dog.txt cat.txt pony.txt bear.txt
a dog.txt
a cat.txt
a pony.txt
a bear.txt

Since we are going to test out extracting the tar, we will go ahead and clean up the files that we put in the tar.

$ rm dog.txt cat.txt pony.txt bear.txt

With all the ceremony of making sure we have a tar file to experiment with out of the way, it is time to fire up our Erlang shell, and call erl_tar:extract/1.

erl_tar:extract("animal_sounds.tar").
% ok

That seemed straight forward enough, so let’s see if we have our files extracted back out at the command prompt.

$ ls dog.txt cat.txt pony.txt bear.txt
bear.txt cat.txt  dog.txt  pony.txt
$ rm dog.txt cat.txt pony.txt bear.txt

And since we saw them, we will go ahead and remove them to get back to a clean state.

erl_tar:extract/2

Erlang also has a erl_tar:extract/2, which allows us to give options to the extraction process, by passing a list as its second argument.

We can have erl_tar:extract/2 extract the files and tell it to be verbose, and then follow that up with another extraction, where we specify that we not only want it to be verbose, but don’t overwrite any files that are already there.

erl_tar:extract("animal_sounds.tar", [verbose]).
% x /Users/proctor/tmp/dog.txt
%
% x /Users/proctor/tmp/cat.txt
%
% x /Users/proctor/tmp/pony.txt
%
% x /Users/proctor/tmp/bear.txt
%
% ok
erl_tar:extract("animal_sounds.tar", [verbose, keep_old_files]).
% x /Users/proctor/tmp/dog.txt - exists, not created
%
% x /Users/proctor/tmp/cat.txt - exists, not created
%
% x /Users/proctor/tmp/pony.txt - exists, not created
%
% x /Users/proctor/tmp/bear.txt - exists, not created
%
% ok

And yet again, we swing back to the command prompt to remove the extracted files.

$ rm dog.txt cat.txt pony.txt bear.txt

Next we extract animal_sounds.tar.gz by passing the atom compressed in the list of options.

erl_tar:extract("animal_sounds.tar.gz", [verbose, compressed, keep_old_files]).
% x /Users/proctor/tmp/dog.txt
%
% x /Users/proctor/tmp/cat.txt
%
% x /Users/proctor/tmp/pony.txt
%
% x /Users/proctor/tmp/bear.txt
%
% ok

And sometimes when working with a tar file in your program, you don’t want to have to do all the management of the files on the filesystem just to read the contents of a tar file, so there is even an option to keep it all in memory.

erl_tar:extract("animal_sounds.tar.gz", [verbose, compressed, keep_old_files, memory]).
% {ok,[{"dog.txt",<<"woofn">>},
%      {"cat.txt",<<"meown">>},
%      {"pony.txt",<<"sparklen">>},
%      {"bear.txt",<<"Wocka Wocka Wocka!n">>}]}

When passing the memory option, the return value of erl_tar:extract/2 becomes an tuple of the status, and a list of tuples composed of the filename, and the contents of the file as a Binary for each file in the tar that was extracted.

If an error occurs on extraction to memory, for example we forget to pass the compressed option to a compressed tar file, it returns an error tuple.

erl_tar:extract("animal_sounds.tar.gz", [verbose, memory]).
% {error,eof}

There are quite a bit more options that erl_tar:extract/2 can take as well, so I highly recommend checking out the documentation for the full list of options.

–Proctor

Ruby Tuesday – String#codepoints

Today’s Ruby Tuesday is on String#codepoints.

String#codepoints returns an Array of Integers that represent the values of the characters in the string.

"Hello, World!".codepoints
# => [72, 101, 108, 108, 111, 44, 32, 87, 111, 114, 108, 100, 33]
"rn".codepoints
# => [13, 10]

For ASCII characters this is what we would expect when looking at an ASCII chart of characters to the decimal value that represents them, but it also works on extended character sets such as Unicode.

"こんにちは世界".codepoints
# => [12371, 12435, 12395, 12385, 12399, 19990, 30028]
"💾".codepoints
# => [128190]

As String#codepoints returns an Array, we can also use the standard methods from the Enumerable module to modify a String.

"HAL".codepoints.map{|codepoint| codepoint + 1}.reduce("", &:<<)
# => "IBM"

And it even works on Unicode characters as well.

"💾".codepoints.map{|codepoint| codepoint + 1}.reduce("", &:<<)
# => "💿"

–Proctor

Erlang Thursday – erl_tar:create/2

Today’s Erlang Thursday is on erl_tar:create/2.

erl_tar:create/2 creates a tar file with a given filename and adds the given list of filenames to the archive.

erl_tar:create/2 takes two arguments, the first is a filename to write to, and the second argument is a list of filenames to add to the tar file.

First, we will open up a new OS shell session and create some files to add to a new tar file.

$ echo "foo" > foo.txt
$ echo "bar" > bar.txt
$ echo "baz" > baz.txt
$ ls
bar.txt  baz.txt  foo.txt  test.tar

Now that we have some files to archive, we can open up a new erl session, and create a new tar file named test.tar.

erl_tar:create("test.tar", ["foo.txt", "bar.txt", "baz.txt"]).
% ok

That looks like it worked; so let’s go to a OS shell, and inspect the resulting file for the filename we gave to erl_tar:create/3.

$ tar -tf test.tar
foo.txt
bar.txt
baz.txt

And yes, tar can read that file and tells us that the three files we added are indeed part of the tar file.

erl_tar:create/3

Erlang also provides erl_tar:create/3 that takes a options list as it’s last argument.

We will create a new file, with the same contents, and pass in that we want this tar file to be compressed, and to be verbose with what it is doing as well.

erl_tar:create("options.tar.gz", 
               ["foo.txt", "bar.txt", "baz.txt"],
               [compressed, verbose]).
% a foo.txt
% a bar.txt
% a baz.txt
% ok

Again, let’s switch back to our OS shell, and inspect the resulting file.

$ tar -tf options.tar.gz
foo.txt
bar.txt
baz.txt

And let’s test it to see if it was considered compressed by gzip.

$ gzip --test options.tar.gz
$

And there we go, gzip considers this a compressed file with integrity. So let’s take a look at the size difference between the two tar files we created.

$ ls -l test.tar options.tar.gz
-rw-r--r--  1 -------  -----    154 Sep XX HH:MM options.tar.gz
-rw-r--r--  1 -------  -----  10240 Sep XX HH:MM test.tar

And looking at the filesize we can see that it is definately compressed, as options.tar.gz is two orders of magnitude smaller than test.tar.

Creating a file that already exists

As we just created test.tar and saw it had the contents, let’s see what happens when we call create on a file that already exists, by passing the same filename with a empty list of files.

erl_tar:create("test.tar", []).
% ok

And we take a look at the contents, we can see the original tar has been replaced.

$ tar -tf test.tar
$

This tells us that erl_tar:create/2 will create a tar file and overwrite the existing file, and doesn’t error out if the file already exists (assuming the user the shell is running has access to write to that file/directory).

Creating a tar for a path that doesn’t exist

If we give a bad path for a file, we can see that erl_tar:create/2 will return a error tuple, with the filename and reason for the failure.

erl_tar:create('/path/does/not/exist.tar', []).
% {error,{'/path/does/not/exist.tar',enoent}}

Other Potential Gotchas

First, the documentation states that it takes filename()s as arguments, but the documentation page for erl_tar does not specify on that page what a filename data type is.

If you use atom()s for the filename, you are going to get an error like the one below that I was getting at first, before using string()s for the filenames.

erl_tar:create('test.tar', ['foo.txt', 'bar.txt', 'baz.txt']).
** exception error: no function clause matching filename:join([]) (filename.erl, line 392)
     in function  erl_tar:split_filename/4 (erl_tar.erl, line 423)
     in call from erl_tar:create_header/3 (erl_tar.erl, line 352)
     in call from erl_tar:add1/4 (erl_tar.erl, line 305)
     in call from erl_tar:foreach_while_ok/2 (erl_tar.erl, line 940)
     in call from erl_tar:create/3 (erl_tar.erl, line 114)

Second, according to the Limitations section of the erl_tar documentation page, filenames should be less than 100 characters for maximum compatability across different systems and version of the tar program.

Lastly, it is on us the user to include the file extension when specifing the filename, as erl_tar:compress/2 does not manage the extension for us.

–Proctor

Ruby Tuesday – String#bytesize

Today’s Ruby Tuesday takes a look at String#bytesize.

String#bytesize returns how many bytes the string uses to represent itself. For standard ASCII characters, the result of bytesize is the same as the length.

"Hello, World!".bytesize
# => 13
"Hello, World!".length
# => 13

But if we go to Google Translate, have it translate “Hello, World” into Japanese, and take that result as the string to call length and bytesize on, we see they can be drastically different.

"こんにちは世界".bytesize
# => 21
"こんにちは世界".length
# => 7

And if you take a string that is just an emoji, say the floppy disk (💾), and call bytesize on that string, you get a value of 4 returned, where the length of the string is only 1.

Why is this important?

The big reason is that you can start to see how much space strings can take up in your storage mechanism of choice (e.g. memory, database, or flat files), and how that can cause issues if you are just checking the length of a string for max size of a field, which may be under the available bytes for storage, but the number of bytes in the string would be too big.

–Proctor

Erlang Thursday – erl_tidy:file/1

Today’s Erlang Thursday takes a look at the erl_tidy module in Erlang, and we start with erl_tidy/1.

erl_tidy:file/1 takes a filename, and “tidies” up and pretty prints the sourcecode in the file specified.

As I have a quick FizzBuzz solution hanging around in a tmp directory, I will just start with that file, and see what erl_tidy:file/1 does to that file, as I am pretty sure it is not completely pretty.

-module(fizzbuzz).

-export([fizzbuzz/1]).

fizzbuzz(N) ->
    Translations = lists:map(fun translate/1, lists:seq(1, N)),
    lists:foreach(fun(Item) -> io:format("~s~n", [Item]) end, Translations).

translate(N) when N rem 3 =:= 0 andalso N rem 5 =:= 0 ->
   'FizzBuzz';
translate(N) when N rem 3 =:= 0 ->
   'Fizz';
translate(N) when N rem 5 =:= 0 ->
   'Buzz';
translate(N) ->
   integer_to_list(N).

We open up a new Erlang shell and call erl_tidy:file/1 with the fizzbuzz.erl.

erl_tidy:file("fizzbuzz.erl").
% fizzbuzz.erl:6: replacing call to `lists:map/2' with a list comprehension.
% fizzbuzz.erl:6: changing application of implicit fun to direct local call.
% ok

Looks like it didn’t like the way I was using map in the original file, and it changed the way I was calling translate/1.

Let’s close out the shell, and see what the directory looks like now.

ls -l
total 16
-rw-r--r--  1 proctor  staff  402 Sep  9 22:06 fizzbuzz.erl
-rw-r--r--  1 proctor  staff  405 Sep  9 22:05 fizzbuzz.erl.bak

Let’s take a look at fizzbuzz.erl.bak and see what is in that file, mainly to confirm that it is the content that was originally in fizzbuzz.erl.

-module(fizzbuzz).

-export([fizzbuzz/1]).

fizzbuzz(N) ->
    Translations = lists:map(fun translate/1, lists:seq(1, N)),
    lists:foreach(fun(Item) -> io:format("~s~n", [Item]) end, Translations).

translate(N) when N rem 3 =:= 0 andalso N rem 5 =:= 0 ->
   'FizzBuzz';
translate(N) when N rem 3 =:= 0 ->
   'Fizz';
translate(N) when N rem 5 =:= 0 ->
   'Buzz';
translate(N) ->
   integer_to_list(N).

That does indeed look like the original contents of fizzbuzz.erl.

Now let’s take a look at the updated fizzbuzz.erl file to see what the result of erl_tidy:file/1 is.

-module(fizzbuzz).

-export([fizzbuzz/1]).

fizzbuzz(N) ->
    Translations = [translate(V1) || V1 <- lists:seq(1, N)],
    lists:foreach(fun (Item) -> io:format("~s~n", [Item])
		  end,
		  Translations).

translate(N) when N rem 3 =:= 0 andalso N rem 5 =:= 0 ->
    'FizzBuzz';
translate(N) when N rem 3 =:= 0 -> 'Fizz';
translate(N) when N rem 5 =:= 0 -> 'Buzz';
translate(N) -> integer_to_list(N).

As the message said, we have a list comprehension instead of map, and it is calling the translate/1 function directly, as mentioned in the two output messages.

It also put the end of the fun passed to lists:foreach/2 on a new line, and the list Translations on a new line as well.

And finally, the last three function clauses of translate/1 were combined into single line functions, and the trailing newline at the end of the file was removed.

According to the documentation page, had the fizzbuzz.eerl file had any unused functions, those would have also have been tidied up, along with “updating obsolete constructs and function calls, etc.”

All in all, a nice little tool of an Erlang application that can help your code keep a certain style and cleanliness to it.

–Proctor

Ruby Tuesday – String#each_char

Today’s Ruby Tuesday covers String#each_char.

String#each_char takes a block, and passes each character in the string to the block.

"hello".each_char{|char| print char.upcase}; puts
# HELLO
# => nil

Note that String#each_char does not modify the original string, and that any intended side effects done in the block must be captured some how if desired.

"hello".each_char{|char| char.upcase}
=> "hello"

If the string is empty, the block is never invoked, as there are no characters in the string to call the block with.

"".each_char{|char| puts "xxx"}; puts "done"
# done
# => nil

If no block is given, String#each_char returns an enumerator, which opens up all of the other methods that Enumerable provides.

"hello".each_char.map{|char| char.upcase}.join
# => "HELLO"
"hello".each_char.select{|char| ["a", "e", "i", "o", "u"].include?(char)}.join
# => "eo"
 "hello".each_char.reject{|char| ["a", "e", "i", "o", "u"].include?(char)}.join
# => "hll"

But then again, String provides a method chars which returns all of the characters in the string as an array.

"hello".chars
# => ["h", "e", "l", "l", "o"]
"".chars
# => []

Which means we also get the full Enumerable on that as well.

"hello".chars.select{|char| ["a", "e", "i", "o", "u"].include?(char)}.join
# => "eo"
"hello".chars.reject{|char| ["a", "e", "i", "o", "u"].include?(char)}.join
# => "hll"

Erlang Thursday – c:pid/3

Today’s Erlang Thursday is a short one on c:pid/3.

c:pid/3 takes the three different parts of a pid as its arguments, and returns a constructed Pid type corresponding to those values.

We’ll call self to get a pid that we know is good, and we can use that to compare the result of calling c:pid/3.

self().
% <0.42.0>
c:pid(0, 42, 0).
% <0.42.0>
self() =:= c:pid(0, 42, 0).
% true

Why is this useful? Sometimes when inspecting what is going on in a live system there are certain calls in Erlang that expect a pid() type, and not just the pid numbers.

c:regs().
%
% ** Registered procs on node nonode@nohost **
% Name                  Pid          Initial Call                      Reds Msgs
% application_controlle <0.7.0>      erlang:apply/2                     463    0
% code_server           <0.19.0>     erlang:apply/2                  128774    0
% erl_prim_loader       <0.3.0>      erlang:apply/2                  163760    0
% error_logger          <0.6.0>      gen_event:init_it/6                220    0
% file_server_2         <0.18.0>     file_server:init/1                 448    0
% global_group          <0.17.0>     global_group:init/1                 59    0
% global_name_server    <0.13.0>     global:init/1                       51    0
% inet_db               <0.16.0>     inet_db:init/1                     206    0
% init                  <0.0.0>      otp_ring0:start/2                 3398    0
% kernel_safe_sup       <0.27.0>     supervisor:kernel/1                 58    0
% kernel_sup            <0.11.0>     supervisor:kernel/1              49109    0
% rex                   <0.12.0>     rpc:init/1                          35    0
% standard_error        <0.21.0>     erlang:apply/2                       9    0
% standard_error_sup    <0.20.0>     supervisor_bridge:standar           41    0
% user                  <0.24.0>     group:server/3                      36    0
% user_drv              <0.23.0>     user_drv:server/2                17940    0
%
% ** Registered ports on node nonode@nohost **
% Name                  Id              Command
% ok
erlang:is_process_alive(c:pid(0, 6, 0)).
% true

So let’s see what happens when we try to feed it something to break it, but in a meaningful way.

c:pid(0, 0, 0).
% <0.0.0>
c:pid(111110, 0, 1111110).
% ** exception error: bad argument
%      in function  list_to_pid/1
%         called as list_to_pid("<111110.0.1111110>")
%      in call from c:pid/3 (c.erl, line 424)

So it looks like the pid <0.0.0> is a valid pid, but when we throw it something else, we see it is trying to call list_to_pid.

So let’s take a quick look at list_to_pid.

erlang:list_to_pid("<0.42.0>").
% <0.42.0>
c:pid(0, 42, 0) =:= erlang:list_to_pid("<0.42.0>").
% true

So it looks like c:pid/3 is a wrapper function around list_to_pid that builds the 3 parts of a pid into a string, and then gets a pid() type from that call.

–Proctor

Ruby Tuesday – String#casecmp

Today’s Ruby Tuesday covers String#casecmp.

String#casecmp is a kind of oddly named method, as what it does is a case insensitive string comparison.

The return value of String#casecmp is either -1, 0, or 1, depending on if the item casecmp is being called on is less than, equal or greater than the string passed in as an argument.

"foobar".casecmp("FOOBAR")
# => 0
"abcdeft".casecmp("ABCDEFG")
# => 1
"abcdefg".casecmp("ABCDEFG")
# => 0
"A".casecmp("b")
# => -1
"a".casecmp("B")
# => -1
"z".casecmp("A")
# => 1
"Z".casecmp("a")
# => 1
"z" <=> "A"
# => 1
"A" <=> "Z"
# => -1
"a" <=> "Z"
# => 1
"Z" <=> "a"
# => -1

This can be handy if you are trying to match two strings by doing a downcase or upcase on the strings, as well as being more clear about what you are trying to accomplish with the comparision.

You can also take advantage of using String#casecmp if you even need to do sorting of items by their name regardless of case.

["foo", "a", "Z", "Foo", "buzz", "FOO"].sort do |a, b|
  result = a.casecmp(b)
  if (result == 0)
    result = a <=> b
  end
  result
end
# => ["a", "buzz", "FOO", "Foo", "foo", "Z"]

–Proctor