Monthly Archives: August 2014

Shelling out in Ruby

One of the things that is nice about Ruby is the ability to use it for scripting. Ruby makes it nice and easy to shell out to run standard *nix commands.

The problem is, the nice simple way that seems to be the most common, is also the least safe. I’m looking at you backticks.

#!/usr/env ruby

result = `gerp -e 'some regex' foo`
puts "<time to process result>"
puts "All Good"

And when we run the above, we see that our script completed, even though we really bombed out trying to run gerp instead of grep.

sh: gerp: command not found
<time to process result>
All Good

Oops.

Sure we get a warning gerp: command not found, but the script still proceeds to plow ahead and do any other side effects that it is setup to do, although something went wrong earlier.

This has become one of the biggest thorns in my side at work.

Lucky the technical solution to this is straight forward, so I want to share it with readers of this blog so you can stop making the mistake of using backticks to shell out commands, and use a solution that does not cover up issues in production.

#!/usr/env ruby

require 'open3'

def execute_syscall(cmd)
  results, error, status = Open3.capture3(cmd)
  raise error unless status.success?

  results
end

result = execute_syscall "gerp -e 'some regex' foo"
puts "<time to process result>"
puts "All Good"

First we require open3. Open3 allows us to be able to capture the results, the error stream, and the status. This way, we can check if the status of the command was anything other than success, and if not, we raise the error we get from STDERR.

Now when we run it, the script stops in its tracks. Not only that, but our script returns a failure error code as well.

> ruby script_demo.rb
script_demo.rb:7:in `execute_syscall': sh: gerp: command not found (RuntimeError)
	from script_demo.rb:12:in `&lt;main&gt;'
> echo $?
1

That way we can know that something went wrong in our program, especially when it is setup as a cron job or as some other unsupervised task.

Hope this can save you some headaches and frustration on your end as well.

–Proctor

Global Day of Coderetreat 2014 – Dallas/Fort Worth

It is that time of year again. The time when the Global Day of Coderetreat date has been announced.

This year, the Global Day of Coderetreat is November 15th of 2014.

And with that, I am pleased to announce the Dallas/Fort Worth area’s participation in Global Day of Coderetreat.

For those who are unfamiliar with Coderetreat, it is a day of practice where developers of all levels and languages get together to engage in deliberate practice. The day is spent in multiple 45 minute sessions, working on the problem of Conway’s Game Of Life. During the sessions the goal is to pair program and use TDD to drive to a solution, while focusing on The Four Rules of Simple Design. At the end of each session, everybody deletes their work, and then has the opportunity to share an insight that they had during the previous session. As the day goes on, constraints get added to the sessions, with the goal to make you think about solving the problem in a different way.

I encourage anybody in the Dallas/Fort Worth area to sign up while space is available, and if you are located somewhere else, check the events listing on the Coderetreat site, and see if there is an event in your area. If not, I highly encourage you to get one setup for your area.

–Proctor

Currently Executing Queries in Vertica

As mentioned in my last post – Timestamps in Vertica – we had a query which caused our cluster of servers to run at over 90% CPU utilization, when the normally in 10-20 percent range. This query was slowing down all of the other queries that were running at the time, even to the point where the query SELECT 1 FROM dual was taking 1200+ milliseconds.

In this case, the trick to finding the culprit was to find the queries that had been running for a long period of time.

SELECT
    *,
    (start_timestamp - statement_timestamp()) AS current_duration
FROM
    query_requests
WHERE
    is_executing = 't'
ORDER BY
    start_timestamp;

We look at the query_requests table, and find those queries that are still executing, and sort them by their start time. I also added in a column current_duration so we could easily determine how long the queries have been running.

With that we found a query that was the culprit, killed that job, and the database cluster recovered quickly.

Hope this can help someone else when they have some runaway queries they need to find.

–Proctor