PullMonkey Blog

VIN API - ignore checksum validation

It's been mentioned a few times that there should be a way to skip the checksum validation. This means if the check digit calculation generally fails, VIN API will continue to process the VIN to see if there is a "like" VIN match. So here it is:

VIN API - fair pricing model

It can't be a shock to any of you that after many months (wait - more like a full year) of running VIN API completely free and out-of-pocket, it is time to monetize. As I am sure most of you know, the back-end data is quite pricey. For example, a monthly subscription for a "lite" data set (your typical year, make, model, engine type, etc) at one of the better known distributors would run you about $260.00 a month for ~800 VIN decodings. This is some very expensive data at 32.5 cents a VIN. Some of you that use our API are running 50,000+ VINs through in a weekend, that is $16,000! Wow!

Ok, so what did we decide to do? Well first, and most importantly, we decided to keep the service up and running; for a while there it was looking pretty bleak and if you take a second to look at the numbers (financially) you can understand why. Anyway, to be as fair as possible, we did not feel this could be monetized as a subscription-based product, some months you need 500,000 VINs and some only a 1,000. Looking at the data for year, we had many questions - what plan would you pick? should the plan rollover it's unused decodings?

We discovered a lot of complexity in the subscription model, so we decided to setup the pay as you go plan, where you buy your decodings at various bulk levels. For instance, you could buy in groups of 1,000 or 10,000 or 100,000 such that you would realize savings on a price-per-VIN basis the larger the group you purchase. There is no use policy either, so you can sit on the VINs for as long as you want, or you can even buy decodings the day (even the minute) before you need them. No monthly credit card bill, no rollovers, just simple "buy what you need," and if you buy in bulk, you save.

Another consideration in doing this was having to maintain soft limits. For example, if the user purchases the 5,000 VINs per month plan, and the their site does well this month and they need 6000 VINs, should they have chosen the 10,000 VIN plan? We didn't think so, we figured no hard limit, just soft limits, and after the soft limit is hit, we would charge the additional VINs at the current rate per VIN to the user's next month's bill. We tried to explain this to a few current users that are helping us come up with reasonable rates and they were not all that thrilled about it. So we had a problem, we did not want users to pay for what they did not need, but at the same time we did not want to cut users off in the middle of a month when they hit their limit. That entire idea had to be scrapped and along with it went the idea behind the subscription plan itself. We like it though, it's simple now - buy your decodings and use your decodings, we will email you if you are getting low and may want to add decodings to your account.

Subscription models are great for static resources and static services, but not a single one of our customers decodes the same number of VINs each month, so it just won't work. So onward and upward, pay as you go and regardless of how the other API providers offer their service, we are excited to be a little different and lot cheaper as you will soon find out.

Enjoy!

POST OFC Graph as Image

I was asked recently (well sort of) to give an example of saving an image to the server. If you look at teethgrinder's example for this, you will see that he has made available an external interface to do just that - POST your graph as png raw data to your server for storage. This has many benefits such as saving the image for use in a PDF report or for printing, since we know at times it is a bit troublesome to print the embedded flash object.

I think the main problem people are having with this is the receiving of the image data post - see the upload_image method below. Also, teethgrinder's example never really says where to make the post_image() call. So I touch on both in the code below.

Here is an example of the png that is saved when I did this for the chart in the previous example:




Well, let's just get right in to the code.

The controller contains the same code as my last post with only a few minor changes to the index method and the addition of the upload_image method.
In the controller, I have this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

class TestItController < ApplicationController
  def index
    # note the user of open_flash_chart_object_from_hash instead of just open_flash_chart_object
    # this allows you to pass in the id of the div you want the the chart to be in
    # this is useful for when we need to findSWF by this id
    @graph = open_flash_chart_object_from_hash("/test_it/chart", :div_name => "my_chart")
  end

  # added to recieve the post data for the OFC png image of the OFC graph
  def upload_image
    name = "tmp_image.png" || params[:name]
    # the save_image method that is provided by the OFC swf file sends raw post data, so get to it like this
    data = request.raw_post
    File.open("#{RAILS_ROOT}/tmp/#{name}", "wb") { |f| f.write(data) } if data
    render :nothing => true
  end

  def chart
    # same code from here - http://pullmonkey.com/2010/01/05/open-flash-chart-ii-x-axis-date-and-time/ 
    ...
  end
end

So just note the use of open_flash_chart_object_from_hash() in the index method, this way we can pass in the id of the div.

In the view, I have this:

1
2
3
4
5

<%= javascript_include_tag 'swfobject.js' %>
<%= @graph %>
<%= save_as_image("http://localhost:3000/test_it/upload_image?name=tmp.png", :id => "my_chart") %>
<br/>
<%= button_to_function "Save Image", "post_image()" %>

Really the only difference from what we would normally have in our view is that I am using the save image setup method that was added to the open flash chart ruby on rails plugin in the last couple hours (as of this post). The save_image method takes some arguments, mainly the url to post the image data to and the id of the chart we setup in the controller.

Open Flash Chart II - X Axis Date and Time

I was asked how to display date and time for the x axis as seen in this teethgrinder example - So here it goes.

Here is the graph we are after in this example:

More Open Flash Chart II examples.

And here is the code (the controller):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

class TestItController < ApplicationController
  def index
    @graph = open_flash_chart_object(550,300,"/test_it/chart")
  end
  def chart
    data1 = []
    data2 = []
    year = Time.now.year

    31.times do |i|
      x = "#{year}-1-#{i+1}".to_time.to_i
      y = (Math.sin(i+1) * 2.5) + 10

      data1 << ScatterValue.new(x,y)
      data2 << (Math.cos(i+1) * 1.9) + 4
    end

    dot = HollowDot.new
    dot.size = 3
    dot.halo_size = 2
    dot.tooltip = "#date:d M y#<br>Value: #val#"

    line = ScatterLine.new("#DB1750", 3)
    line.values = data1
    line.default_dot_style = dot

    x = XAxis.new
    x.set_range("#{year}-1-1".to_time.to_i, "#{year}-1-31".to_time.to_i)
    x.steps = 86400

    labels = XAxisLabels.new
    labels.text = "#date: l jS, M Y#"
    labels.steps = 86400
    labels.visible_steps = 2
    labels.rotate = 90

    x.labels = labels

    y = YAxis.new
    y.set_range(0,15,5)

    chart = OpenFlashChart.new
    title = Title.new(data2.size)

    chart.title = title
    chart.add_element(line)
    chart.x_axis = x
    chart.y_axis = y

    render :text => chart, :layout => false
  end
end

And in your view (index.html.erb):

1
2
3
4

<script type="text/javascript" src="/javascripts/swfobject.js"></script>
<%= @graph %>

Good Luck!

Open Flash Chart II for Ruby on Rails - Lug Wyrm Charmer

A long time overdue, but I've managed to get everything updated to the new version of Teethgrinder's open flash chart.

I've also started tagging everything, so if you notice any problems trying to do anything from Teethgrinder's examples, then first check that you are using the latest (as of now, that is Lug Wyrm Charmer) - http://github.com/pullmonkey/open_flash_chart/tree/LugWyrmCharmer.

Make sure you are using the latest swf either from the plugin assets directory or from Teethgrinder's downloads.

Using Tumblr as a CMS

Thought you all might like this - http://blog.skizmo.com/post/308406755/use-tumblr-as-your-cms

It is something we sort of dreamed up and it works great as a partial CMS - very much like SimpleCMS where you can specify what exactly on the page needs to be managed by a CMS. This allows you to mix your CMS static content with your dynamic content.

THINning it out

Been having problems with swap space and memory on my slicehost servers.  And it is all apache's and mongrel's fault.  That used to be the cool combination and now it is an ugly, sluggish beast.  Just recently, I switched to nginx (to replace apache) and thin (to replace mongrel).  So far so good, major speed improvements and definitely memory consumption improvements.

I started out by switching everything over the nginx while keeping the mongrels alive, that was actually pretty easy.  Information was available everywhere.

Thinning everything via capistrano took a while, that wasn't as well documented.  Thin was documented, capistrano was documented, but easy solutions as to how to combine the two were difficult to find.

Here's the solution I was able to come up with -

Capistrano

My config for using mongrel used to look something like this -

Now that we are moving from mongrel to thin, no need for two lines in particular, one being the line that requires mongrel_cluster recipes and the other that sets the mongrel_cluster yaml config path.  A third line changes from mongrel_cluster.yml to thin_cluster.yml.  You get something like this:

Now we need to implement what mongrel recipes was doing for us, start, stop and restart but in terms of thin (added this to the bottom of my deploy.rb):

Here's what my thin_cluster.yml looks like:

That's it and it has worked out nicely so far.

Open Flash Chart II - fully automated

Just as an attention grabber - we are going after this example in this article:

Keeping up

Ok, seeing that the php versions of open flash chart and open flash chart swf files continually change along with with the API (not saying this is a bad thing), I wanted to come up with an even more abstract solution. The goal is to not have to worry when the swf file is released with the latest set of graphs or changes its API. I simply don't want to worry about this method or that method, or this class or that class.

Feedback

This article will sort of act as a tutorial for those interested in metaprogramming and as a set of instructions for those looking to experiment with the latest version of the OFC II Rails Plugin that I am currently toying with. I would like to hear feedback, but just remember that phase 1 of this release will be very basic, meaning none of the ajaxy stuff. It will come, just not yet.

Let's see what we can get away with

I am already using method_missing() for pretty much everything in the OFC II Rails Plugin that is being used now. But every time new classes are added, I have to sit down and basically convert the php class to ruby - just plain tedious, not really what I had planned when I started all this. Ok, so method_missing() was great, but let me introduce (or possibly reintroduce) you to const_missing(), basically method_missing() but instead of methods, we can create classes or modules or other objects on the fly. This will definitely help when the php version gets a new class. Instead of getting hounded to update the rails version to be 100% like the php version, everything will just work, no updates to code required. Well, we hope ! So check this out:



Here is what we did with method_missing():

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

module OFC
  class Base
    def method_missing(method_name, *args, &blk)
      case method_name.to_s
      when /(.*)=/   # i.e., if it is something x_legend=
        # if the user wants to set an instance variable then let them
        # the other args (args[0]) are ignored since it is a set method
        self.instance_variable_set("@#{$1}", args[0])
      when /^set_(.*)/
        # backwards compatible ... the user can still use the same set_y_legend methods if they want
        self.instance_variable_set("@#{$1}", args[0])
      else
          if inst = self.instance_variable_get("@#{method_name}")
            inst
          else
            # if the method/attribute is missing and it is not a set method then hmmmm better let the user know
            super
          end
      end
    end
  end
end

This just basically allows me to do this:

1
2
3
4
5
6
7
8
9
10
11
12
13

  class Foo < OFC::Base
  end

  foo = Foo.new

  foo.some_random_attribute = "Hello"  #=> "Hello"
  foo.some_random_attribute  #=> "Hello"
  foo.some_random_undefined_attribute  #=> Method Missing error (calls super)

  # too be like php, for easier conversion
  foo.set_some_random_attribute("Good Bye")  #=> "Good Bye"
  foo.some_random_attribute  #=> "Good Bye"

Along the same lines, I have created an initialize method that takes any argument hash of variable/value pairs and calls variable=() which is handled by method missing as we saw above:

1
2
3
4
5
6
7
8

  class Foo < OFC::Base
  end

  foo = Foo.new(:x_axis => 5, :min => 10, :max => 90, :steps => 5, :elements => ["one", "two"])
  
  foo.x_axis #=> 5
  foo.min #=> 10

Ok, so on to const_missing() and what we can do with that:

1
2
3
4
5
6

  def OFC.const_missing(const)
    klass = Class.new OFC::Base
    Object.const_set const, klass
    return klass
  end

This says that any undefined (missing) constant of OFC should be defined as a new class that inherits from OFC::Base.

So when we say OFC::Foo, that has not been defined, so we will get back class OFC::Foo < OFC::Base;end; which will give us the initialize() method and method_missing() method from above. Let's see how this works:

1
2
3
4
5
6
7
8
9
10
11

  line = OFC::Line.new(:values => [1,2,3,nil,nil,5,6,7])
  line.values #=> [1,2,3,nil,nil,5,6,7]
  line.some_random_variable = "Hello" #=> "Hello"
  line.some_random_variable #=> "Hello"

  stacked_bar_chart = OFC::BarStack.new
  stacked_bar_chart.values = []
  stacked_bar_chart.values << [2,3,4]
  stacked_bar_chart.values << [5, {"val" => 5, "colour" => "#ff0000"}]
  stacked_bar_chart.keys = [{ "colour" => "#C4D318", "text" => "Kiting", "font-size" => 13 } ...]

So it all sort of came together right there. I've shown you all the code that comes with the Rails Open Flash Chart plugin now. No more definining idividual classes, no more trying to keep up with the never ending php version, and no more late nights converting php to ruby (!). About dang time.



Ok, but this is just the beginning, nothing has been set in stone, so like I said, give me your feedback, what works for you and what does not. And, hopefully, I will have solutions for you or you for me.

Example with new version (test version)

I am using rails 2.3.2, but I don't think it will matter what version you are using.

Create your new rails project

1
2
3
4
5

# create a new rails project 
> pullmonkey$ rails testing_it
#<Bunch of stuff is created ....>
> pullmonkey$ cd testing_it/

Install the plugin from the test branch

Note the -r test in this next step. The new version (test version) I am playing with is under the test branch and -r says what branch to pull from.

Also, you can use git:// instead of http:// below, but depending on your firewall restrictions http:// will probably work out best for you.

1
2
3

> pullmonkey$ ./script/plugin install http://github.com/pullmonkey/open_flash_chart.git -r test
# <Bunch more stuff ...>

Create a controller to play in

1
2
3

> pullmonkey$ ./script/generate controller test_it
# <And more stuff >

Get our assets

1
2
3
4
5
6
7

# first we will get swfobject.js
> pullmonkey$ cp vendor/plugins/open_flash_chart/assets/javascripts/swfobject.js public/javascripts/
# next the open flash chart swf (GET whatever is the latest version), right now that is here: http://teethgrinder.co.uk/open-flash-chart-2/open-flash-chart.swf
> pullmonkey$ cd public/
> pullmonkey$ wget http://teethgrinder.co.uk/open-flash-chart-2/open-flash-chart.swf
> pullmonkey$ cd ..

Edit our controller

Notice here that I just include one of the many examples from the plugin's examples directory. Definitely more to follow.

One thing you will notice about the examples, is that the php code is in the comments, so you can see how I would convert from the php examples to ruby. Please feel free to add your own examples, just fork the project.

1
2
3
4
5
6
7
8
9
10

> pullmonkey$ vi app/controllers/test_it_controller.rb
# mine looks like this:
class TestItController < ApplicationController
  include OFC::Examples::AreaHollow

  def index
    @graph = open_flash_chart_object(600,300, "/test_it/area_hollow")
  end
end

Edit our view

1
2
3
4
5

> pullmonkey$ vi app/views/test_it/index.html.erb
# mine looks like this:
<%= javascript_include_tag 'swfobject' %>
<%= @graph %>

Start 'er up

1
2
3
4
5

> pullmonkey$ ./script/server

# browse to the test_it index
http://localhost:3000/test_it

Our example

Spreadsheet Gem - data may have been lost

I've been using the spreadsheet gem lately for a couple projects I am working on to modify existing spreadsheets. I have quite often stumbled upon this error when opening modified spreadsheets in excel:

File error: data may have been lost

Like most microsoft errors, it was useless and the spreadsheet came up just fine. But that error was just so annoying, other spreadsheet applications (open office, excel on mac) opened without any problems. So after quite a bit of hacking and digging around, I finally tried setting the encoding, which defaults to UTF-8. Well it just so happens that the spreadsheet being modified was encoded with UTF-16LE.



So part one of my solution became this:

1
2

Spreadsheet.client_encoding = 'UTF-16LE'

Then doing a little more digging I decided that this would be a better long-term solution:

1
2
3

book = Spreadsheet.open spreadsheet_file
Spreadsheet.client_encoding = book.encoding

Well, hopefully it wasn't just me and someone will be able to save a bit of time with this.

Can you read this?

Got an email today, I have seen it before and I am sure it has been going around for years. This time, I thought that I would do an exercise and create a plugin that duplicates what I found in this email. See for yourself.



Here is the email I got:

./script/plugin install http://github.com/pullmonkey/can_you_read_this.git

And use it like this:

#in your views
<%= can_you_read_this("hello, can you read this?") %>

Have fun.