Erlang Thursday – string:tokens/2

Today’s Erlang Thursday is string:tokens/2.

string:tokens/2 takes a string as the first argument, and a list of separators to split the string on, and returns a list of token strings.

string:tokens("foo", "").
% ["foo"]
string:tokens("banana", "a").
% ["b","n","n"]
string:tokens("It was the best of times, it was the worst of times", " ").
% ["It","was","the","best","of","times,","it","was","the",
%  "worst","of","times"]

If consecutive separators appear in the string they will be treated as a single separator, and no empty strings will be returned.

string:tokens("Mississippi", "s").
% ["Mi","i","ippi"]
65> string:tokens("Mississippi", "sp").
% ["Mi","i","i","i"]
string:tokens("Mississippi", "is").
% ["M","pp"]

The order of the separators in the separator list passed to string:tokens/2 does not matter, and can be specified in any order.

string:tokens("Mississippi", "ps").
% ["Mi","i","i","i"]
65> string:tokens("Mississippi", "sp").
% ["Mi","i","i","i"]

And as the separator list is just simply a list of separators, instead of passing a string, the integer values for the characters to use as the separators can be passed as a list, as a list of the integers is the same as a string.

% 115
% 112
[115, 112].
% "sp"
string:tokens("Mississippi", [115]).
% ["Mi","i","ippi"]
string:tokens("Mississippi", [115, 112]).
% ["Mi","i","i","i"]