New package: GetQuandlData

Quandl and tidyverse

Introduction

Quandl is one of the best platforms for finding and downloading financial and economic time series. The collection of free databases is solid and I use it intensively in my research and class material.

But, a couple of things from the native package Quandl always bothered me:

  • Multiple data is always returned in the wide (column oriented) format (why??);
  • No local caching of data;
  • No control for importing error and status;
  • Not easy to work within the tidyverse collection of packages

As you suspect, I decided to tackle the problem over the weekend. The result is package GetQuandlData. This is what it does differently:

  • It uses the json api (and not the Quandl native function), so that some metadata is also returned;
  • The resulting dataframe is always returned in the long format, even for multiple series;
  • Users can set custom names for input series. This is very useful when using along ggplot or making tables;
  • Uses package memoise to set a local caching system. This means that the second time you ask for a particular time series, it will grab it from your hard drive (and not the internet);
  • Always compares the requested dates against dates available in the platform.

Installation

# not in CRAN yet (need to test it further)
#install.packages('GetQuandlData')

# from github
devtools::install_github('msperlin/GetQuandlData')

Example 01 - Inflation in the US

Let’s download and plot information about inflation in the US:

library(GetQuandlData)
library(tidyverse)

my_id <- c('Inflation USA' = 'RATEINF/INFLATION_USA')
my_api <- readLines('~/Dropbox/.quandl_api.txt') # you need your own API (get it at https://www.quandl.com/sign-up-modal?defaultModal=showSignUp>)
first_date <- '2000-01-01'
last_date <- Sys.Date()

df <- get_Quandl_series(id_in = my_id, 
                        api_key = my_api, 
                        first_date = first_date,
                        last_date = last_date, 
                        cache_folder = tempdir())

glimpse(df)
## Observations: 238
## Variables: 4
## $ series_name <chr> "Inflation USA", "Inflation USA", "Inflation USA", "Infla…
## $ ref_date    <date> 2019-10-31, 2019-09-30, 2019-08-31, 2019-07-31, 2019-06-…
## $ value       <dbl> 1.764, 1.711, 1.750, 1.811, 1.648, 1.790, 1.996, 1.863, 1…
## $ id_quandl   <chr> "RATEINF/INFLATION_USA", "RATEINF/INFLATION_USA", "RATEIN…

As you can see, the data is in the long format. Let’s plot it:

p <- ggplot(df, aes(x = ref_date, y = value/100)) + 
  geom_col() + 
  labs(y = 'Inflation (%)', 
       x = '',
       title = 'Inflation in the US') + 
  scale_y_continuous(labels = scales::percent)

p

Beautiful!

Example 02 - Inflation for many countries

Next, lets have a look into a more realistic case, where we need inflation data for several countries:

First, we need to see what are the available datasets from database RATEINF:

library(GetQuandlData)
library(tidyverse)

db_id <- 'RATEINF'
my_api <- readLines('~/Dropbox/.quandl_api.txt') # you need your own API

df <- get_database_info(db_id, my_api)

knitr::kable(df)
code name description refreshed_at from_date to_date quandl_code quandl_db
CPI_ARG Consumer Price Index - Argentina Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1988-01-31 2013-12-31 RATEINF/CPI_ARG RATEINF
CPI_AUS Consumer Price Index - Australia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:46 1948-09-30 2019-09-30 RATEINF/CPI_AUS RATEINF
CPI_CAN Consumer Price Index - Canada Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:46 1989-01-31 2019-09-30 RATEINF/CPI_CAN RATEINF
CPI_CHE Consumer Price Index - Switzerland Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:46 1983-01-31 2019-10-31 RATEINF/CPI_CHE RATEINF
CPI_DEU Consumer Price Index - Germany Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1991-01-31 2019-10-31 RATEINF/CPI_DEU RATEINF
CPI_EUR Consumer Price Index - Euro Area Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1990-01-31 2019-10-31 RATEINF/CPI_EUR RATEINF
CPI_FRA Consumer Price Index - France Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1990-01-31 2019-10-31 RATEINF/CPI_FRA RATEINF
CPI_GBR Consumer Price Index - UK Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1988-01-31 2019-10-31 RATEINF/CPI_GBR RATEINF
CPI_ITA Consumer Price Index - Italy Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:46 2001-01-31 2019-10-31 RATEINF/CPI_ITA RATEINF
CPI_JPN Consumer Price Index - Japan Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:46 1970-01-31 2019-09-30 RATEINF/CPI_JPN RATEINF
CPI_NZL Consumer Price Index - New Zealand Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:46 1988-03-31 2019-09-30 RATEINF/CPI_NZL RATEINF
CPI_RUS Consumer Price Index - Russia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1995-01-31 2019-09-30 RATEINF/CPI_RUS RATEINF
CPI_USA Consumer Price Index - USA Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:46 1913-01-31 2019-10-31 RATEINF/CPI_USA RATEINF
INFLATION_ARG Inflation YOY - Argentina Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1989-01-31 2013-12-31 RATEINF/INFLATION_ARG RATEINF
INFLATION_AUS Inflation YOY - Australia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1949-03-31 2019-09-30 RATEINF/INFLATION_AUS RATEINF
INFLATION_CAN Inflation YOY - Canada Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:46 1990-01-31 2019-09-30 RATEINF/INFLATION_CAN RATEINF
INFLATION_CHE Inflation YOY - Switzerland Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1984-01-31 2019-10-31 RATEINF/INFLATION_CHE RATEINF
INFLATION_DEU Inflation YOY - Germany Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1992-01-31 2019-10-31 RATEINF/INFLATION_DEU RATEINF
INFLATION_EUR Inflation YOY - Euro Area Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:46 1991-01-31 2019-10-31 RATEINF/INFLATION_EUR RATEINF
INFLATION_FRA Inflation YOY - France Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1991-01-31 2019-10-31 RATEINF/INFLATION_FRA RATEINF
INFLATION_GBR Inflation YOY - UK Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1989-01-31 2019-10-31 RATEINF/INFLATION_GBR RATEINF
INFLATION_ITA Inflation YOY - Italy Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 2002-01-31 2019-10-31 RATEINF/INFLATION_ITA RATEINF
INFLATION_JPN Inflation YOY - Japan Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1971-01-31 2019-09-30 RATEINF/INFLATION_JPN RATEINF
INFLATION_NZL Inflation YOY - New Zealand Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 2001-03-31 2019-09-30 RATEINF/INFLATION_NZL RATEINF
INFLATION_RUS Inflation YOY - Russia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1996-01-31 2019-09-30 RATEINF/INFLATION_RUS RATEINF
INFLATION_USA Inflation YOY - USA Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-11-23 02:08:47 1914-01-31 2019-10-31 RATEINF/INFLATION_USA RATEINF

Nice. Now we only need to filter the series with YOY inflation:

idx <- stringr::str_detect(df$name, 'Inflation YOY')
df_series <- df[idx, ]

and grab the data:

my_id <- df_series$quandl_code
names(my_id) <- df_series$name
first_date <- '2010-01-01'
last_date <- Sys.Date()

df_inflation <- get_Quandl_series(id_in = my_id, 
                                  api_key = my_api,
                                  first_date = first_date,
                                  last_date = last_date)

glimpse(df_inflation)
## Observations: 1,303
## Variables: 4
## $ series_name <chr> "Inflation YOY - Argentina", "Inflation YOY - Argentina",…
## $ ref_date    <date> 2013-12-31, 2013-11-30, 2013-10-31, 2013-09-30, 2013-08-…
## $ value       <dbl> 10.95, 10.54, 10.55, 10.49, 10.55, 10.61, 10.46, 10.34, 1…
## $ id_quandl   <chr> "RATEINF/INFLATION_ARG", "RATEINF/INFLATION_ARG", "RATEIN…

And, finally, an elegant plot:

p <- ggplot(df_inflation, aes(x = ref_date, y = value/100)) + 
  geom_col() + 
  labs(y = 'Inflation (%)', 
       x = '',
       title = 'Inflation in the World',
       subtitle = paste0(first_date, ' to ', last_date)) + 
  scale_y_continuous(labels = scales::percent) + 
  facet_wrap(~series_name)

p

Avatar
Marcelo S. Perlin
Associate Professor of Finance

Related

comments powered by Disqus