How to Make Conditional Requests to Github’s API Using Octokit

For the last month, my team and I have been working on Gitbo, an app for finding open source software issues on Github and providing incentives for improvement by having users issue monetary bounties for solutions.

One of the challenges we had to overcome was managing our calls to the Github API. Github’s starting limit on API requests for developers is 5000 per hour. Gitbo needs to pull all sorts of data from Github’s API about developer’s repositories, their related issues and comments, as well as user details.

Furthermore, we wanted to keep repository and issue information as fresh as we could in Gitbo. This required frequent calls to the Github API for every record in our database containing information from Github. It wasn’t long before we started hitting the limit, which became a huge roadblock in development as well as a large red flag of future scalability problems for our app in production.

The Github API documentation outlines how to make conditional requests by adding an If-Modified-Since header to the HTTP request to the API which will return a 304 Not Modified HTTP response if the resource hasn’t been modified since that time. This was a great solution for our problem because any 304 responses from Github do not count against our rate limit. If the API resource has been modified since the time you requested, the API returns the normal JSON response.

The question was how to add this into our app. We had already implemented a fantastic open source API wrapper called octokit, maintained by pengwynn who has been a super helpful resource for our group. My fellow team members Adam and Kevin found a pull request to Octokit from bratsche which adds support for making conditional requests. However, Wynn decided not to merge into Octokit, preferring a middleware solution, so they merged the new commits from bratche/octokit into Adam’s fork of octokit.

You can view the additional source code that adds the support in the merge commit.

If you’d like to make conditional requests in your application using octokit simply add octokit into your Gemfile with Adam’s repo as the source:

1
gem 'octokit', :github => 'ajonas04/octokit'

Make sure to bundle install or bundle update depending on whether you already have pengwynn’s original octokit loaded.

Now with conditional requests, you can add a :since option into all requests to Github that will make the request conditional, returning the full JSON response of the resource if its Last-Modified header in the conditional request’s response is later than the datetime query string you sent.

In order for it to work, you’ll need to send the query string to Github in RFC 1123 Time Format. Luckily, Ruby’s Time library has a method httpdate which will convert a Ruby Datetime or Time object into a string with this format.

Let’s see it in action, I’ll make a conditional request with the current time as the :since parameter. Since the resource will probably not have been modified in the time between the time object’s creation and when Github receives the request, we’ll see the benefits for our rate limit:

1
2
3
4
5
6
7
8
client = Octokit::Client.new(:login => username, :password => password)
client.rate_limit_remaining
#=> 5000
current_time = Time.now.httpdate
client.repositories("joshrowley", :since => current_time)
#=> nil # this nil tells us we received a 304 response
client.rate_limit_remaining
#=> 5000

How to Get Started Writing Tests With Minitest

Untitled

Testing let’s you check on the health of your code, making sure it’s performing well. Photo by Jeff Werner

Up until today, all of my project work at Flatiron has been focused on writing application code. Today, I started learning about how to write the companion test suites. It’s a bit intimidating, so here’s how to get started.

Why Testing is Important

Writing test suites is important because it helps ensure that your program does what you think it does. Your tests define your expectations for your program. When you run your test suite, your tests should pass if your program’s behavior matches these expectations, and fail if the reality is different.

Testing becomes very powerful once you start coding using a test driven development approach. Rather than writing tests for your existing code, you write your tests before anything else. Define the expected behavior in the test, and then write the necessary code to get the test to pass. Once, the test is passing, refactor your code to remove repetition and improve code reability. As you continue to build a large collection of tests, your full test suite should warn you if new code starts to cause bugs somewhere else. These early warning flags help keep your code stable and minimize bugs.

What is Minitest?

Minitest is a lightweight testing library for Ruby. Minitest used to be a separate gem, but as of Ruby 1.9 it’s a part of the standard library that comes with Ruby. Require the library in any Ruby file and it adds all kinds of methods that make writing tests easier.

How to get started writing tests with Minitest?

For example, let’s say I’m writing a Flashcard class for an application for studying Spanish. Before writing any part of the Flashcard class, I’ll want to create tests for how I expect the Flashcard class to behave.

I’ll first create a file for my test suite, test.rb. Within this file, I’ll want to make sure that the Minitest library is loaded before the tests are run. This is done by requiring the library:

require 'minitest/spec'
require 'minitest/autorun'

We’ll also want to load the file that will contain the code we’re testing. In the same directory, let’s say flashcard.rb is where I’m going to start writing the class definition. I’ll load it in through the require relative method:

require_relative 'flashcard.rb'

Now, I’ll write my tests. I should write tests for every expected behavior for the class. How do I write a test?

describe Flashcard do
end

Step 1: Use Mintest’s describe method to create a block where all your tests relating to the Flashcard class will go. All your tests will go between the do and the end keywords.

describe Flashcard do
  it "can create an instance of the Flashcard class" do
  end
end

Step 2: Use the it method and pass in a string that describes the expected behavior. This will be the text that is printed when the test is failing to help you identify which functionality of your code is not behaving correctly. Be sure to pick a good name.

describe Flashcard do
  it "can create an instance of the Flashcard class" do
    flashcard = Flashcard.new
  end
end

Step 3: Set up what needs to be in place before you make an assertion about the test. In this case, I’m testing whether I can create an instance of the Flashcard class, so I’ll want to set the stage by creating an instance.

describe Flashcard do
  it "can create an instance of the Flashcard class" do
    flashcard = Flashcard.new
    flashcard.must_be_instance_of Flashcard
  end
end

Step 4: Write an assertion about the expected behavior. Minitest has easy to understand methods such as must_be_instance_of that determine whether your test passed (whether the assertion is true), or failed (assertion returned false).

Continue to write out all your expectations, until you have a sizeable list. Here are some example tests, you’ll probably have more tests than this, but note that there are tests are very basic stuff like whether the class even exists, up to whether the class adds a card into a user’s favorites.

describe Flashcard do

    it "can create an instance of the Flashcard class" do
      flashcard = Flashcard.new
      flashcard.must_be_instance_of Flashcard
    end

    it "has a word attribute" do
      flashcard = Flashcard.new
      flashcard.word = "hola"
      flashcard.word.must_be == "hola"
    end

    it "should respond to add_to_favorites method" do
      flashcard = Flashcard.new
      flashcard.must_respond_to add_to_favorites
    end

    it "should add favorite cards to a user's favorites" do
      #TODO: write test once I figure out user class spec
    end
end

I run the tests by running the test file through Terminal: ruby test.rb. This is where a testing tool like Minitest comes in handy. The test file will print to the terminal any errors and whether the code from flashcard.rb passes the test. If it doesn’t pass, Minitest will print the actual value and the expected value that were inputted into that test’s assertion. When I run the above test suite, the first error I receive is:

/Users/josh/Desktop/test.rb:6:in <main>': uninitialized constant Flashcard (NameError)`

My suite describes a class Flashcard, but Ruby can’t find the class. In accordance with a test driven approach, I now add code into flashcard.rb, but only enough to get rid of that first error. In this case, all I need to do is define the class in flashcard.rb:

class Flashcard
end

Now if I run test.rb again, I get the following:

Run options: --seed 19557

# Running tests:

.EE.

Finished tests in 0.001041s, 3842.4592 tests/s, 960.6148 assertions/s.

  1) Error:
test_0003_should_respond_to_add_to_favorites_method(Flashcard):
NameError: undefined local variable or method `add_to_favorites' for #<#<Class:0x007f9e8c0c65e8>:0x007f9e8c062a70>
    test.rb:21:in `block (2 levels) in <main>'

  2) Error:
test_0002_has_a_word_attribute(Flashcard):
NoMethodError: undefined method `word=' for #<Flashcard:0x007f9e8c061be8>
    test.rb:15:in `block (2 levels) in <main>'

4 tests, 1 assertions, 0 failures, 2 errors, 0 skips

As you can see, Minitest creates a nicely formatted results output for you. Your next step is to tackle fixing the next failing test. What’s great about writing these tests is that it forces you to explicitly define what your application should do before you write the code. Sometimes, writing tests for one class will help you figure out what other classes you need. For example on the last test, I want to add cards to a user’s favorites. Obviously, I’ll need to now write tests for a User class, and then also eventually make some design decisions on how cards are added to a user.

Tests make you break the problem down into the smallest possible chunks, keep you focused on one small problem at a time, and help prevent feature creep and getting sidetracked. Plus, there’s an addictive reward you feel when you get a test to pass, even if it’s super basic, that’ll keep you motivated.

The Fog of Programming

Avi talks about the “fog of programming”. There are so many different technologies at play in a web application, it’s all overwhelming at first. Not only that, but since all the technologies interact with each other, you need to be comfortable with how all of them work.

As a result, new programmers feel like they’re in a fog cloud, where they kind of understand how things work and interact with each other.

Then one day, you’ll finally learn something, and you’ll go back to those fuzzy concepts and suddenly a ray of clarity will strike your mind and things start to come together more.

I’m really starting to understand classes, and how damn useful they are. I sort of understood them before, but I finally “saw the light” once I started connecting the dots between scope, instance variables, and class methods.

A couple days ago, variable scoping really started to click in my head and feel natural and intuitive. Now I see instance variables, instance methods, and class methods in a whole new light.

It’s quite a rewarding feeling.

Earlier in the semester, I wrote a very simple jukebox example, but have now rewritten it using object oriented code.

Here’s the original:

And here’s the object oriented version:

Breakable Toys: Improve Your Programming Through Small, Low-stakes Projects

I was perusing Speaker Deck and found this great deck by Aaron Kalin on learning programming through creating breakable toys.

There are some great suggestions he makes for beginning programmers who are trying to grow. A couple of the points are ones that I hear from Avi all the time.

What’s a breakable toy?

As a newcomer to the software development field, the principles Aaron mentions are important to keep in mind for any new developer and I see how they relate to my experience:

  1. “Go Simple”

    When given a coding challenge or project to implement, I tend to overthink it, or get too ambitious. I try to design my code to be open for all these awesome features I’d like to add. I try to implement everything at once, instead of doing small, basic, iterative steps. Also, when thinking of new projects I want to start, I try to think of next Facebook or amazing web app, but in order to grow as a developer I need to think of what small tools I could build that would be a great learning experience.

  2. “They don’t need to be perfect”

    For a lot of beginners, myself included, there’s an intense urge to “get it right”. There must be a perfect way of doing something. As I’ve become more immersed in the coding world, I’ve noticed that this is a destructive instinct. First, there are a million different ways to tackle a problem. Second, I need to being comfortable with sharing imperfect code with others. The worst thing is to work on code in isolation until the day I think it’s “perfect”. That day will most likely never come, and the code you wrote will never be used or shared.

  3. “Consider open-sourcing them. It may help someone else learn”

    Pair programming at Flatiron school has been such a great learning experience. It’s been extremely beneficial for me to code with another person. Going back to the previous points, even if your program is super simple and imperfect, share it with people! It’s easy to forget that at one point, a former version of myself wouldn’t have been able to do it, and would have searched for the answer online.

  4. “Re-inventing the wheel is ok”

    With all the gems available on Ruby, it’s amazing that I can easily add some amazing functionality to my applications, even without a ton of experience. I now want to try my hand at building my own version of the sqlite3-ruby or nokogiri gems. Even though it’s not necessary, it gives me projects I can work on, and gain further insight.

4 Tips for Solo GitHubbers: How to Code With People for the First Time

Coding Together Photo by hackNY

I used Git collaboratively for the first time exactly one week ago on Day 1 of Flatiron School. It wasn’t pretty, you can read a little about it in my first post.

I had some experience using Git while doing Michael Hartl’s Rails Tutorial in my spare time, but I had never used it with other people. Git is very simple when it’s just you. I was pretty much only doing commits and pushing to the remote, with the occasional branch.

This is one of the biggest limitations facing beginners trying to learn how to become professional developers on their own, whether through books or online resources. You must learn how to use Git with other people! That much is very apparent from my first week at Flatiron. If a coding academy isn’t an option for you, find a Meetup for beginners. Convince your friends to learn it with you. Find somebody.

Once you got your collaborators, here are some tips I’d like to share based on my first experience of being a newbie lone wolf Git user being thrown into my first group Git collaboration:

  1. Establish team’s workflow process and best practices first.

    It’s really important that everyone understands conceptually how your repo is organized, and who should be committing where. Do not rush this step! If someone is a little shaky on the foundations, take the time to get them up to speed. Even someone with the best of intentions can cause havoc on the rest of the group’s work if they misunderstand your group’s workflow.

  2. Make sure you’re working on the correct branch.

    I pulled down the master branch, and created my own feature branch. However, at some point I checked out the master branch and didn’t notice. I lazily pushed to GitHub without specifying a specific branch (I assumed was in my feature branch), and as a result committed a major party foul by pushing to the master branch. If you’re making commits on the wrong branch, you’re setting yourself up for confusion when trying to merge feature branches together, or merging them back to master.

  3. Do not run any commands if you don’t understand what it does.

    Not only that, but do you understand its consequences in your specific context? I was super eager to get started with git. I didn’t remember everything, so I asked my fellow classmates what commands to try when I ran up against a wall. Or, I would try stumbling around with git commands until I got what I wanted. Do not do this! Git is not IRB! You can’t just play around with it. If you’re stuck, take as much time as you need to research and figure out what you need to do, or ask someone who would know. Otherwise, you can screw up your repo big time.

  4. One mistake can proliferate exponentially into a huge mess.

    Unfortunately, my only advice is to try to not make any mistakes, especially when merging, rebasing, pulling, and pushing. Take a moment to step back and doublecheck that you are using the correct command, in the correct branch, to the correct remote. Ask yourself, when I hit enter, what will this do? If you’re not 100%, don’t do it.

SQL: Schemin’ With Schemas

Today was spent working on databases. I had a lot of fun writing SQL queries. We were tasked to create the schema for a simple quiz app. Here’s the schema:


CREATE table user (
    id INTEGER PRIMARY KEY,
    name TEXT
);

CREATE table quiz (
    id INTEGER PRIMARY KEY,
    name
);

CREATE table question (
    id INTEGER PRIMARY KEY,
    content TEXT,
    quiz_id INTEGER
);

CREATE table choice (
    id INTEGER PRIMARY KEY,
    content TEXT,
    correct BOOLEAN,
    question_id INTEGER
);

CREATE table answer (
    id INTEGER PRIMARY KEY,
    user_id INTEGER,
    question_id INTEGER,
    choice_id INTEGER
);

With this schema, we had to write some queries and insert sample data. I pair programmed with Matt, and the last challenge was a bit tricky for us, but it was a fun challenge to complete. It feels really good to be learning so much so quickly. Create a stats.sql that will show the percentage (if you want to try to use MATH functions within SQL) or the total right answers to each question. This will need to use joins too and might get complicated.


SELECT question.content, (COUNT (*) * 100.0) / (SELECT COUNT (*) FROM answer WHERE question_id = question.id)
FROM question
INNER JOIN choice
ON question.id = choice.question_id
INNER JOIN answer
ON question.id = answer.question_id
WHERE choice.correct = 1 AND
question.id = 1 AND
answer.choice_id = choice.id;

We then iterate through all the questions replacing the question.id value in the where clause. That outputs the following data with the percentage of responses that were correct:


How do you initialize a Git repository?|100.0
What git command stages changes?|50.0
How do you make a commit once changes are staged?|100.0
What command do you use to see whether there are any untracked files in your repository?|100.0
How do you create a new branch and switch to it with one line?|50.0
How do you create a table in SQL?|100.0
How do you remove a table in SQL?|50.0
How do you write a query that returns all rows from a table?|50.0
What is the difference between an inner join and an outer join?|50.0
How do you add a row to a table?|50.0

Hello World

Hi everyone, my name is Josh Rowley, and I’m a student at the newly created Flatiron School. It’s been two days since we started, and I’ve already felt that I’ve gained more understanding of web development in the past two days then any of the self study I did before I started in the months leading up to it.

The most helpful thing to have in order to learn how to code, at least for me, is to be working with a group with the same goals and mission as yourself. That’s what we have here at the school. It also allows us to work in more realistic environment. The benefits of learning in a team for a beginner cannot be understated. Everyone has different strengths and thought processes, and putting them together helps us solve the challenges of development in faster and newer ways.

Of course, working in the school has also presented its own challenges compared to self study. I thought I knew git pretty well from studying previously and working on my own projects, but I had never actually used it to collaborate on a project with anyone else.

Day one, Avi had us create profile pages for the school website with HTML and CSS. Sounds simple enough, but we were all going to do it at the same time using the same github repo, by forking and creating our own feature branches. At the end, we were going to merge all of our separate work back into the master branch.

We were divided into teams of four, and we started to plan our workflow and collaborate on how the HTML layouts and CSS styling would look like. We thought we had a handle on how git worked, but after a couple of hours, we had way too many branches with common files spread out across our disparate local git repos with different versions across all of them. It was a mess, and we realized that we were a bit ambitious for the first day.

It also showed me that the biggest thing that I’m going to learn here is being able to work as a team on software. Avi emphasized this point several times and it was quite obvious after our first day that that was the case. Being able to communicate and be on the same page for workflow and planning is critical and I’m excited to learn these skills here. It’s definitely something that cannot be learned on your own.