Entries in Ruby (2)

Thursday
Feb192009

Weekend Reading

Weekend Reading

Barbara and I went down to the Tattered Cover on Sunday for some reading time. It was a beautiful day, and I like the TC for a couple of reasons:

  1. Big bookstores ("Tattered Cover", "Powell's" in Portland, the "Harvard Coop" in Cambridge) have a charm all their own
  2. Chain bookstores (modern Barnes and Nobel, the almost-extinct Borders) all seem to carry the same books at all locations. What's the chance of finding something new and different?

Anyway, the TC didn't disappoint, and I came away with three books, each with a neat angle on modern computing

The Annotated Turing

First up is The Annotated Turing, but Charles Petzold. This is one of the most remarkable books I've ever read, in that it takes one of the crowning intellectual efforts of the 20th Century, Alan Turing's "On Computable Numbers, with an Application to the Entscheidungsproblem", and highlights it with mathematical historical and cultural annotations.

You have to admire any book that tries, in its first two chapters, to explain number theory to its readers. Talk about a "thankless task" but Petzold does a nice job of it.

By the time you get to page 33's "and at some point the Heisenberg Uncertainty Principle kicks in, and then we really can't be sure of anything any more", you're hooked. Awesome and fun tell-all of the founding document of computer science.

Viral Spiral

Next up is Viral Spiral, by David Bollier. Tim Berners-Lee originally created what would become the Internet as a sharing platform for academic researches. Some sharing -- by 2003 the Internet was being used by 600 million people worldwide, and Viral Spiral describes the general creative commons and open source worlds -- focused around what Bollier calls the "Great Value Shift" in how valuable things are created for commerce and culture.

In commercial software development this cultural shift is still taking place; the ability to scale at almost zero marginal cost is probably the core driver behind what's been called "Web 2.0." Hard to do better than R. Buckminster Fuller, who said:

You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.

Developing Facebook Platform Applications with Rails

Facebook matters, and even more than Google and Amazon, Facebook memes dominate 21st century software development thinking right now:

  • 400,000 registered developers?
  • 24,000 applications?
  • 140 new applications per day?

Numbers like that talk, and Michael Mangino's book DFPAwR goes under the hood of this technology phenomenon and describes the building of applications on Facebook's platform.

Facebook is something remarkable; successful, global Level-3 computing platforms generally only come up every couple of decades (IBM 360, Microsoft Windows and the Internet are probably the only other 3 in the last 50 years), so for Facebook to have made the progress they've made since 2007 is mindbending.

My interest here is pragmatic -- how could it be possible to build such a widespread platform in so little time?, and my reading is also with the hope of figuring out how much Level 2 / Level 3 platform-ness we can build into Magellan.

Platform birth is rare, but it's compelling when it happens. I'm open to ideas, and as we come up with things I'll keep you posted...

Sunday
Feb152009

Adventures in Code: the Google Analytics API

Google Analytics

Google is huge and well-known, and they swallowed the old Urchin code and company to provide web analytics. You can't beat the introductory cost (free, as in beer), and they offer nice graphs and a pretty analytical interface. What's not to like? Well, it's possible that Google doesn't offer every form of analytics you need -- then you need an API, to do your own analytics-synthesis. You need a Google Analytics API...

The API

What API? It's easy to be glib and say No Google Analytics API? Not Problem!, but that's really not a good enough answer. The problem here is that analytics is kind of a sensitive area for Google -- they like to determine just-how-much you know about your website (not more). I'd love a rich interface with REST access, but that's not Google.

Google doesn't even hesitate -- their own web site proclaims their nothingness, API-wise

How We Get There -- First Try: Rugalytics

The best thing I've found so far to break the barrier is Rugalytics, which appears to give a nice Ruby interface to Google Analytics. Ruby is a bit funky as a language, but it is elegant and is terrific for scripting -- I'll have more on the Ruby language in a later post.

I installed it on my Mac, and proceeded to put it through its paces. Here's what I found out...

OK -- code / session time

First lets get get libraries and get started.

JKR-MBP:rugalytics johnrepko$ irb


>> require 'rubygems'; require 'rugalytics'
=> true

Now we log in -- through Rugalytics to Google

>> Rugalytics.login 'jkrepko', '????????'


=> #<Google::Base:0x18503bc @sid="DQAAAHAAAAAd0EaDrxbSRKo6EjFbdUSltb2I1gjSlvDCfQvHVVVIlBJmRui-
rpbnERXvcbt3eYV25YrJScDgK2foNWMcGmoSoVZnLCMYXS8vAUKXJAX5Kttu
MD-pCYOADKRl18aRbxzJAIUnfXpKSaUIUh-gqWdz", @email="jkrepko", @password="?????????">

Let's link to the site info for my techblog:

>> profile = Rugalytics.find_profile('tmn-techblog.com')


=> #<Rugalytics::Profile:0x17dbaa8 @profile_id=15171077, @account_id=7527070, @name="tmn-techblog.com">

... and make a quick grab of pageview information:

>> profile.pageviews :from=>'2009-02-17', :to=>'2009-02-18'


pageviews_graph
=> 43

43 pageviews? Not so many. Let's look by date:

>> report = profile.pageviews_report :from=>'2009-02-17', :to=>'2009-02-18'


pageviews_graph
=> #<Rugalytics::Report:0x17921dc @base_url="tmn-techblog.com", @pageviews_graph=#<Rugalytics::Graph:0x178dc04 
@points_by_day=[[Tue, 17 Feb 2009, 15], [Wed, 18 Feb 2009, 
28]], @name="Pageviews", @to=Wed, 18 Feb 2009, @from=Tue, 
17 Feb 2009, @points=[15, 28]>, name"Pageviews for all 
visitors", end_dateWed, 18 Feb 2009, start_dateTue, 17 Feb 2009

>> report.pageviews_by_day


=> [[Tue, 17 Feb 2009, 15], [Wed, 18 Feb 2009, 28]]

Not a bad start -- let's see what canned reports are available:

>> profile.report_names


=> ["ad_versions_report", "adwords_report", "all_sources_report", "average_pageviews_report",
"bounce_rate_report", "browsers_report", "campaigns_report", "colors_report", 
"content_by_title_report", "content_drilldown_report", "content_report", "dashboard_report", 
"depth_of_visit_report", "direct_sources_report", "entrances_report", "exits_report", "flash_report", 
"geo_map_report", "hostnames_report", "java_report", "keyword_position_report", "keywords_report", 
"languages_report", "length_of_visit_report", "loyalty_report", "networks_report", "os_browsers_report",
 "pageviews_report", "platforms_report", "recency_report", "referring_sources_report", "resolutions_report",
 "search_engines_report", "speeds_report", "time_on_site_report", "top_content_detail_keywords_report", 
"top_content_detail_navigation_report", 
"top_content_detail_path_report", "top_content_detail_sources_report", "top_content_report",
 "traffic_sources_report", "unique_visitors_report", "visitor_types_report", "visitors_overview_report", 
"visits_report"]

Terrific -- let's see what browsers have been used on the blog site

>> report = profile.browsers_report :from => '2009-02-17', :to=>'2009-02-20'
>> for r in report.items
>> print r.visits, ' ', r.browser, '\n'
>> end


17 Safari
7 Internet Explorer
4 Mozilla
3 Firefox

Great - not a bad set -- now let's dig into a report:

>> report = profile.top_content_detail_keywords_report(:url=>'tmn-techblog.com/archives')
items


pageviews_graph
=> #<Rugalytics::Report:0x18756e4 @base_url="tmn-techblog.com", @pageviews_graph=#
<Rugalytics::Graph:0x1863ae8 @points_by_day=[[Mon, 19 Jan 2009, 0], [Tue, 20 Jan 2009, 0], [Wed, 21 Jan 2009, 0], 
[Thu, 22 Jan 2009, 0], [Fri, 23 Jan 2009, 0], [Sat, 24 Jan 2009, 0], [Sun, 25 Jan 2009, 0], [Mon, 26 Jan 2009, 0], 
[Tue, 27 Jan 2009, 0], [Wed, 28 Jan 2009, 0], [Thu, 29 Jan 2009, 0], [Fri, 30 Jan 2009, 0], [Sat, 31 Jan 2009, 0], 
[Sun, 01 Feb 2009, 0], [Mon, 02 Feb 2009, 0], [Tue, 03 Feb 2009, 0], [Wed, 04 Feb 2009, 0], [Thu, 05 Feb 2009, 0], 
[Fri, 06 Feb 2009, 0], [Sat, 07 Feb 2009, 0], [Sun, 08 Feb 2009, 0], [Mon, 09 Feb 2009, 0], [Tue, 10 Feb 2009, 0], 
[Wed, 11 Feb 2009, 0], [Thu, 12 Feb 2009, 0], [Fri, 13 Feb 2009, 0], [Sat, 14 Feb 2009, 0], [Sun, 15 Feb 2009, 0], 
[Mon, 16 Feb 2009, 0], [Tue, 17 Feb 2009, 0], [Wed, 18 Feb 2009, 0], [Thu, 19 Feb 2009, 0]], @name="Pageviews", 
@to=Thu, 19 Feb 2009, @from=Mon, 19 Jan 2009, @points=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]>, name"Entrance Keywords:", end_dateThu, 19 Feb 2009, items[], 
start_dateMon, 19 Jan 2009

Let's see what the top items this week are:

>> top_items_over_week = report.items.sort_by{|i| i.unique_pageviews.to_i}.reverse


=> [#<Rugalytics::Item:0x16343f8 @path="/", @dollar_index="0.0", @bounce_rate="0.3125", 
@unique_pageviews="16", @pageviews="38", @percentage_exit="0.42105263157894735", 
@avg_time_on_page="186.95454545454547", @url="http://tmn-techblog.com/", @page="/">, #<Rugalytics::Item:0x1632940 
@path="/past/2008/12/16/welcome_to_simplelog_this_is/", @dollar_index="0.0", @bounce_rate="0.0", 
@unique_pageviews="4", @pageviews="4", @percentage_exit="0.0", @avg_time_on_page="8.5", 
@url="http://tmn-techblog.com/past/2008/12/16
/welcome_to_simplelog_this_is/", @page="/past/2008/12/16/welcome_to_simplelog_this_is/">, 
#<Rugalytics::Item:0x1630f00 @path="/past/tags/AWS", @dollar_index="0.0", @bounce_rate="0.0", 
@unique_pageviews="1", @pageviews="1", @percentage_exit="0.0", @avg_time_on_page="617.0", 
@url="http://tmn-techblog.com/past/tags/AWS", @page="/past/tags/AWS">]

Enough for now -- let me run with a bit more data, and I'll dig deeper into the reports and check on performance and data transfer / transformation options.

dynamic language scripting Google analytics Ruby API Rugalytics