Use Go to get a list of repositories from GitHub
In this post I’m going to ask to the GitHub API information about public repositories, using just the Go net/http
stdlib package, without any additional library.
GitHub has a nice public API called Search, which you can inspect at https://developer.github.com/v3/search/#search-repositories.
In particular, I’m interested in getting the Go repos with more than 10k stars at this point in time.
We can pass it a query to get exactly what we want. Looking at the Search API docs we’ll have to make an HTTP GET request to https://api.github.com/search/repositories
passing the query ?q=stars:>=10000+language:go&sort=stars&order=desc
: give me the repositories with more than 10k stars, with language Go, sort them by number of stars.
Simplest snippet: we use http.Get
from net/http
and we read everything that’s returned using ioutil.ReadAll
:
package main
import (
"io/ioutil"
"log"
"net/http"
)
func main() {
res, err := http.Get("https://api.github.com/search/repositories?q=stars:>=10000+language:go&sort=stars&order=desc")
if err != nil {
log.Fatal(err)
}
body, err := ioutil.ReadAll(res.Body)
res.Body.Close()
if err != nil {
log.Fatal(err)
}
if res.StatusCode != 200 {
log.Fatal("Unexpected status code", res.StatusCode)
}
log.Printf("Body: %s\n", body)
}
It’s not much readable, as you can see. You can paste what you get using an online formatter to understand the actual structure.
Let’s now process the JSON and generate a nice table from it.
If you’re new to the JSON topic, check out my preliminary guide on how to process JSON with Go.
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
"text/tabwriter"
"time"
)
// Owner is the repository owner
type Owner struct {
Login string
}
// Item is the single repository data structure
type Item struct {
ID int
Name string
FullName string `json:"full_name"`
Owner Owner
Description string
CreatedAt string `json:"created_at"`
StargazersCount int `json:"stargazers_count"`
}
// JSONData contains the GitHub API response
type JSONData struct {
Count int `json:"total_count"`
Items []Item
}
func main() {
res, err := http.Get("https://api.github.com/search/repositories?q=stars:>=10000+language:go&sort=stars&order=desc")
if err != nil {
log.Fatal(err)
}
body, err := ioutil.ReadAll(res.Body)
res.Body.Close()
if err != nil {
log.Fatal(err)
}
if res.StatusCode != http.StatusOK {
log.Fatal("Unexpected status code", res.StatusCode)
}
data := JSONData{}
err = json.Unmarshal(body, &data)
if err != nil {
log.Fatal(err)
}
printData(data)
}
func printData(data JSONData) {
log.Printf("Repositories found: %d", data.Count)
const format = "%v\t%v\t%v\t%v\t\n"
tw := new(tabwriter.Writer).Init(os.Stdout, 0, 8, 2, ' ', 0)
fmt.Fprintf(tw, format, "Repository", "Stars", "Created at", "Description")
fmt.Fprintf(tw, format, "----------", "-----", "----------", "----------")
for _, i := range data.Items {
desc := i.Description
if len(desc) > 50 {
desc = string(desc[:50]) + "..."
}
t, err := time.Parse(time.RFC3339, i.CreatedAt)
if err != nil {
log.Fatal(err)
}
fmt.Fprintf(tw, format, i.FullName, i.StargazersCount, t.Year(), desc)
}
tw.Flush()
}
The above code creates 3 structs to unmarshal the JSON provided by GitHub. JSONData
is the main container, Items
is a slice of Item
, the repository struct, and inside item, Owner
contains the repository owner data.
Given the HTTP response res
returned by http.Get
we extract the body using res.Body
and we read all of it in body
with ioutil.ReadAll
.
After checking the res.StatusCode
matches the http.StatusOK
constant (which corresponds to 200
), we unmarshal the JSON using json.Unmarshal
into our JSONData
struct data
, and we print it by passing it to our printData
struct.
Inside printData
I instantiate a tabwriter.Writer
, then process and format the JSON data accordingly to output a nice table layout:
I wrote 17 books to help you become a better developer, download them all at $0 cost by joining my newsletter
JOIN MY CODING BOOTCAMP, an amazing cohort course that will be a huge step up in your coding career - covering React, Next.js - next edition February 2025