Author Archives: steven.proctor@gmail.com

Running Clojure shell scripts in *nix environments

I was recently trying to create a basic piece of Clojure code to play with “real-time” log file parsing by playing with futures. The longer term goal of the experiment is to be able to tail -f a log file pipe that into my Clojure log parser as input.

As I wasn’t sure exactly what I would need to be doing, I wanted an easy way to run some code quickly without having to rebuild the jars through Leiningen every time I wanted to try something, in a manner similar to the way I am thinking I will be using it if the experiment succeeds.

I created a file test_input with the following lines:

1 hello
2 test
3 abacus
4 qwerty
5 what
6 dvorak

With this in place, my goal was to be able to run something like cat test_file | parser_concept. After a bit of searching I found the lein-exec plugin for Leiningen, and after very minor setup I was able to start iterating with input piped in from elsewhere.

The first step was to open my profiles.clj file in my ~/.lein directory. I made sure lein-exec was specified in my user plugins as so:

{:user {:plugins [[lein-exec "0.2.1"]
                  ;other plugins for lein
                 ]}}

With this in place I just put the following line at the top of my script.clj file:

#!/usr/bin/env lein exec

I then changed the permissions of script.clj file to make it executable, I was able to run the following and have my code run against the input.

cat test_input | ./script.clj

I will be posting a follow up entry outlining my next step of experimenting with “processing” each line read in as a future.

Remove first and last lines from file in OS X

Just a quick post to help burn this into longer term memory.

Today I was having to check some info in a generated csv file that had a header and footer row. I only wanted the records in between, so I needed to remove the first and last lines of that CSV, after I got the columns I needed.

cut <args> my.csv | tail -n +2 | sed '$d'

The tail -n +2 command starts at the second line and outputs the input stream/file. The sed '$d' command deletes the last line of the file.

Looking for a new job

The company I have been working for had to just go through a round of layoffs, due to not getting one of their main contracts renewed do to lack of funding by the state, and I have been included in that round.

My primary experience has been in C# on the Microsoft .Net framework for the past 10 years. Of those 10 years, 9 of them were working against the same product, and product suite allowing me to learn how some inconsequential decisions are not so inconsequential long term, and the value of care and commitment to a code base can be.

I am thinking that this would be a good opportunity to open myself up in trying to venture into a new language and toolset such as Ruby or Clojure. This is my open announcement for anybody who would like to poach a .Net developer if you are having problems finding developers fluent in your programming language.

–Proctor

DFW Clojure User Group

This is for those of you not on the Clojure Google Group mailing list.

There is now a Clojure User Group for the Dallas/Fort Worth metroplex. Alex Robbins has setup a meetup group which can be found here http://www.meetup.com/DFW-Clojure/.

The kickoff meeting is set for Wednesday, September 19, 2012.

Thanks to Alex and the sponsors of this user group. Let’s make it a good one.

–Proctor

DRM Free Please

I try to be a voracious reader and just over a year ago, I purchased a nook touch. There were three primary drivers for that decision.

First was I wanted to have an e-ink display after sitting and looking at a computer monitor all day, the e-ink is easier on my eyes, especially late at night in bed before I go to sleep.

Second was from the recommendation about epub being better support from publishing perspective from the Pragmatic Programmers when I asked over twitter whether to go with the Kindle or KindleDX. Their reply was that epub is a more open and extensible format to work with and software books tend to come out better in that format.

Third one was that the epub is a more open format than the others I had encountered and the nook supports that format.

On the down side though, the Barnes and Noble store has all of their books with DRM on them.

For using my nook, I will always try and buy from publishers and stores that give DRM free books, and if not, I would rather buy a print version. I could give a number of reasons that I am against DRM, from being free to share books with another person, being free to read my books on whatever device I decide without being losing any content I paid for, lack of trust in major publishers or booksellers being able to survive the shift into digital publishing and taking away my content if they don’t.

I want to give thanks to the following publishers for their support of DRM free publishing, and these publishers will always get my support: The Pragmatic Bookshelf, O’Reilly Media, Manning Publications, and Apress.

As part of this I would also like share the Readers Bill of Rights for Digital Books.

I will gladly support any additional publishers and sellers who put their support behind DRM free digital books.

–Proctor

Clojure function has-factors-in?

Just another quick post this evening to share a new function I created as part of cleaning up my solution to Problem 1 of Project Euler.

Was just responding to a comment on Google+ on my update sharing the post Project Euler in Clojure – Problem 16, and I saw the commenter had his own solution to problem 1. In sharing my solution I realized that I could clean up my results even further, and added a function has-factors-in?. These updates have also been pushed to my Project Euler in Clojure Github repository for those interested.

(defn has-factors-in? [n coll]
  (some #(factor-of? % n) coll))

Where before I had:

(defn problem1
  ([] (problem1 1000))
  ([n] (sum (filter #(or (factor-of? 3 %) (factor-of? 5 %))) (range n))))

It now becomes:

(defn problem1
  ([] (problem1 1000))
  ([n] (sum (filter #(has-factors-in? % [3 5]) (range n)))))

This change makes my solution read even more like the problem statement given.

Your thoughts?

–Proctor

Support for Arrays in Constructors using Castle Windsor

As I mentioned in my post Aspect Oriented Timing in C# with Castle Windsor, we are using Castle Windsor as our Inversion of Container on our current project.

In the process of adding a couple of new features, there were places where the new code had to fit in to existing places where there were a longer (more than simply just a single if/else) set of conditionals. To address the changes that I needed to make, I added another conditional to go with the existing ones. This change was in the mood of make it work, knowing I would come back after it worked to make it right, and that this change would allow me to confirm the smell I was “sure” I was smelling.

One I had it working, I came back to make it right. The route I wanted to go was to emulate the refactoring Replace Conditional with Polymorphism by creating a “handler” abstraction, and pulling the logic for each of the conditionals into a separate implementation of the handler.

My new interface was defined as:

public interface INewHandler
{
  bool CanHandle(ISomeObject obj);
  void Handle(ISomeObject obj);
}

I then moved out the logic in each of the conditionals, as well as the conditionals themselves into new handlers. The conditionals were moved into the CanHandle method, and the body of the conditional was moved to the HandleMethod of the implementation.

With these defined, it became just a matter of using a LINQ query to find either the first, or all items that could handle the object, and then just call the Handle method on the ones that matched the filter.

injectedHandlers.
    Where(h => h.CanHandle(someObj)).
    ForEach(h => h.Handle(someObj));

After having this, all we need now is the ability to have all of the implementations injected into the constructor. This gets us to the title of this post, about having arrays injected into constructors. First we have to configure the container to add a new ArrayResolver as a subresolver.

container.Kernel.Resolver.AddSubResolver(new ArrayResolver(container.Kernel));

After this, I want to pick up all implementations of the INewHandler in all assemblies.

container.Register(
    AllTypes.FromAssemblyInDirectory(new AssemblyFilter("bin")).
    BasedOn&gt;INewHandler>().
    WithService.Base().
    Configure(c =&gt; c.LifeStyle.PerWebRequest)
);

After adding these two to the registration of Castle Windsor, I now get all implementations injected into my constructor when I declare a constructor parameter that is a type of INewHandler[].

Hope someone else can find this helpful.

–Proctor

Project Euler in Clojure – Problem 16

Here is my solution to Problem 16 of Project Euler. As always, my progress you can tracked on GitHub at https://github.com/stevenproctor/project-euler-clojure.

Problem 16 of Project Euler is:

What is the sum of the digits of the number 2^1000

This problem was straight forward since I already had the function digits-of defined from problem8. I was able to be very declarative in my problem, so much so, that it reads as the problem statement you are asked to solve.

(defn problem16
  ([] (problem16 1000))
  ([n] (sum (digits-of (expt 2 n)))))

As always, any feedback you have for me is greatly appreciated.

–Proctor

LaTeX in WordPress

Writing up my post Project Euler in Clojure – Problem 15 I wanted to represent the formulation for combinations in a nice manner. As I previously had found the [sourecode] tags, I figured it was worth a shot at doing some searching to see if WordPress.com supported any forumla formatting plug ins out of the box.

I have known of LaTeX for formulas but as I use a wordpress.com site, I knew it only ships with a limited number of plug ins, so I did a search about mathematical formula formatting in WordPress.com. Through the blog post Math for the Masses, I discovered that WordPress.com does indeed support LaTeX.

In my previous post mentioned above, I had the following formula for combinations:

$\frac{n}{\big((n-k)!*k!\big)}$

So to break that down and outline how I display that lets give the WordPress markup I used to get that:

So to start with we have $latex which begins our LaTex statement, and if you notice we have a $ that signifies the end of the expression. The example on for the LaTeX in LaTeX is coded:

Which results in the following:

$LaTeX$

The next part in the LaTex statement begins the expression, which is the fraction part of the expression where

\frac{numerator}{denominator}

giving:

$\frac{numerator}{denominator}$

The parenthesis are done using big( and big). The last part of the expression is parameters to update the size of the display in WordPress. The WordPress support page for LaTeX discusses the different parameters available for the LaTeX. The parameter s is the parameter to specify the size of the LaTeX, of which I set as 3 which is the numeric representation of LARGE.

The posts mentioned above contained links to other resources for LaTeX. The two additional ones I used were: a LaTeX wiki on The Student Room and The Not So Short Introduction to LATEX 2ε.

Project Euler in Clojure – Problem 15

Here is my solution to Problem 15 of Project Euler. As always, my progress you can tracked on GitHub at https://github.com/stevenproctor/project-euler-clojure.

Problem 15 of Project Euler is to find the starting number with the longest Collatz sequence, summarized from the problem page as:

Starting in the top left corner of a 22 grid,
there are 6 routes (without backtracking) to the bottom right corner.

How many routes are there through a 2020 grid?

I started this problem, by trying to tracing and counting the routes through grids of 2×2, 3×3, and 4×4, and even setled in and did a 5×5 square. Having these numbers, and knowing I had two choices for ever position I was in, except for when the path got to the far edge and bottom, I had a hint at the growth rate of the problem. I tried some powers of 2 with the relationship of the numbers, and some factorials with the numbers. After seeing some possible relationships with the factorials that might be leading me in the right direction, I tried a number of permutation calculations, and the combination calculations. Having seen the numbers show up in different combination results, I then spent time back-calculating from those numbers into my ns, and found that the pattern seemed to match 2n Choose n.

The source code to this was the factorial function:

(defn factorial [n]
  (multiply (range 1M (inc (bigdec n)))))

And, I could have done it recursively, but I figured I would just operate against the sequence of numbers, especially now that the reducers are available in the Clojure 1.5-Alpha 3 release (at the time of this writing) of Clojure. After I get through a few more problems (of which I am working ahead of these posts), I am thinking it would be interesting to run the same Project Euler Problems against 1.4 and 1.5 using the reducers library, just substituting map/reduce for the reduce/combine functionality, and seeing how much effort it takes to move them over, as well as the differences in the timings of the different problems.

The other base function I needed was a combination function:

(defn combination [n k]
  (cond (zero? n) 0
        (zero? k) 1
        :else (/ (factorial n) (* (factorial (- n k)) (factorial k)))))

This function just does the basic calculation for combinations, from the formula:

$\frac{n!}{\big((n-k)! * k!\big)}$

With that, and my stumbling upon the matching of the fact that ${2n}\choose{n}$ is the solution to the number of paths through the square the function problem15 is defined as:

(defn problem15
  ([] (problem15 20))
  ([n] (combination (+ n n) n)))

As always, any feedback you have for me is greatly appreciated.

–Proctor