Understanding Requests

Requests serve as the backbone of web interactions, facilitating communication between various websites, servers, and services. They play a critical role for both users and developers by allowing for the seamless transfer of data. Although many programming languages handle requests consistently, Julia's approach may differ from what you expect if you come from another language. However, using requests in Julia is still quite manageable.

I have been developing a portfolio website in Julia utilizing the Toolips.jl framework. One key feature I wanted to incorporate was the automatic display of all my GitHub repositories in a user-friendly format. To achieve this, we will delve into the HTTP.jl package for making requests and then interpret that data into a usable format in Julia. For those interested in the code, you can find the project notebook here:

Making Requests

Creating requests in Julia using the HTTP.jl package is straightforward. To fetch the repositories from my organization, ChifiSource, which hosts several interesting projects, we would execute the following:

using HTTP

This line results in an HTTP.Response object rather than the actual request data. To access the data, we need to call the body of the response, which is not a String initially. We can convert it to a String with:

data_string = String(response.body)

Next, I'll use the JSON.jl package, which simplifies the process of parsing JSON strings into dictionaries:

using JSON

items = JSON.parse(data_string)

After obtaining the data, I printed one of the dictionaries representing a repository to understand its structure:

println(values(items)[1])

Transforming Data into Julia Types

The raw data can be overwhelming and not very user-friendly. I focused on a few essential attributes that I wanted to extract: name, description, language, stars, and tags. To organize this data, I created a custom data structure:

mutable struct RepoData

name::Any

desc::Any

url::Any

language::Any

stars::Any

tags::Array

end

To populate this structure, I implemented an inner constructor that extracts data from a dictionary. Tags are fetched from a separate URL, as indicated in the response:

tag_url = d["tags_url"]

tag_response = HTTP.request("GET", tag_url)

The tags arrive in a string format resembling an array, requiring a custom function to parse this data into a Julia array:

function string_to_array(s::String)

s = replace(s, "[" => "")

s = replace(s, "]" => "")

split(s, ",")

end

If the tags were integers, we would adapt the function to convert the strings into integers. However, since tags are strings, this is unnecessary for our current task.

The constructor for RepoData is finalized with the following method:

function RepoData(d::Dict)

tag_url = d["tags_url"]

tag_response = HTTP.request("GET", tag_url)

tags = string_to_array(String(tag_response.body))

new(d["name"], d["description"], d["html_url"], d["language"], d["stargazers_count"], tags)

end

Now, we can use a comprehension to create instances of RepoData for each repository:

repository_data = [RepoData(d) for d in items]

Examples of the generated RepoData instances include:

This approach is both efficient and effective!