Solve chapters Importing data from the web Part 1 and 2 at DataCamp.
Read: Chapters 1-3 in An introduction to APIs
Resources:
Write a function
get_pi <- function(start, numberOfDigits){
...
}that calls a-pi and
returns the digits of \(\pi\) from
start to start + numberOfDigits - 1. A sample
call is https://api.pi.delivery/v1/pi?start=1000&numberOfDigits=5.
A TimeEdit-schedule can be obtained in JSON-format by changing
.html to .json in the url, try in a
web-browser with https://cloud.timeedit.net/su/web/stud1/ri107655X95Z06Q6Z96g0Y75y5066Y35Q01gQY6Q55737.html
Part 1: We can use GET to import it to
R
library(httr)
schema_response <- GET("https://cloud.timeedit.net/su/web/stud1/ri107655X95Z06Q6Z96g0Y75y5066Y35Q01gQY6Q55737.json")
schema_json <- content(schema_response, "text")The result can be explored with jsonedit in the package
listviewer
library(knitr)
library(tidyverse)
library(jsonlite)
library(listviewer)
jsonedit(schema_json)We convert it with fromJSON and choose the
reservations
schema_df <- fromJSON(schema_json)$reservationsThe result is now a data.frame with six columns, where
the last column contains a vector in each cell. In order to extract
elements from this column, we use mutate in combination
map_chr. The family of map-functions comes
from the purrr package which is part of the
tidyverse, you will learn more about them later in the
course. Here map_chr(columns, 1) corresponds to
sapply(columns, function (x) x[1]) in base-R.
schema_df %>% mutate(sal = map_chr(columns, 1),
kurs = map_chr(columns, 3),
event = map_chr(columns, 7),
tid = paste(starttime, endtime, sep = " - ")) %>%
select(kurs, datum = startdate, tid, event, sal) %>%
kable() %>%
kableExtra::kable_styling(bootstrap_options = "striped", full_width = FALSE)| kurs | datum | tid | event | sal |
|---|---|---|---|---|
| MT4007 | 2022-11-07 | 08:00 - 12:00 | Föreläsning | Lärosal 1. Albano Hus 1. Vån 2 |
| MT4007 | 2022-11-10 | 08:00 - 12:00 | Föreläsning | Lärosal 4. Albano Hus 1. Vån 2 |
| MT4007 | 2022-11-14 | 08:00 - 12:00 | Föreläsning | Lärosal 1. Albano Hus 1. Vån 2 |
| MT4007 | 2022-11-17 | 08:00 - 12:00 | Föreläsning | Lärosal 5. Albano Hus 1. Vån 2 |
| MT4007 | 2022-11-21 | 08:00 - 12:00 | Föreläsning | Lärosal 5. Albano Hus 1. Vån 2 |
| MT4007 | 2022-11-24 | 08:00 - 12:00 | Föreläsning | Lärosal 1. Albano Hus 1. Vån 2 |
| MT4007 | 2022-11-28 | 08:00 - 12:00 | Föreläsning | Lärosal 5. Albano Hus 1. Vån 2 |
| MT4007 | 2022-12-01 | 08:00 - 12:00 | Föreläsning | Lärosal 1. Albano Hus 1. Vån 2 |
| MT4007 | 2022-12-05 | 08:00 - 12:00 | Föreläsning | Lärosal 5. Albano Hus 1. Vån 2 |
| MT4007 | 2022-12-08 | 08:00 - 12:00 | Föreläsning | Lärosal 1. Albano Hus 1. Vån 2 |
| MT4007 | 2022-12-12 | 08:00 - 12:00 | Föreläsning | Lärosal 5. Albano Hus 1. Vån 2 |
| MT4007 | 2022-12-15 | 08:00 - 12:00 | Föreläsning | Lärosal 4. Albano Hus 1. Vån 2 |
| MT4007 | 2022-12-19 | 08:00 - 12:00 | Föreläsning | Lärosal 1. Albano Hus 1. Vån 2 |
| MT4007 | 2022-12-21 | 09:00 - 11:00 | Föreläsning | Webbaserad undervisning |
| MT4007 | 2023-01-05 | 14:00 - 19:00 | Tentamen (OBS Anmälan i LADOK är obligatorisk!) | *Obestämd skrivsal |
| MT4007 | 2023-01-12 | 09:00 - 15:00 | Redovisning | Lärosal 16. Albano Hus 2. Vån 2 |
Part 2: The schedule for Albano Hus 1, lecture room 1 during the first and second part of the winter semester 2022 can be found on
https://cloud.timeedit.net/su/web/stud1/ri107665X55Z57Q6Z46g0Y60y7026Y31Q05gQY6Q54737.html
If you generate a table at Statistikdatabasen, you
will find a link “API för denna tabell” that gives an url and a query to
be made by POST in order to fetch the table. Try fetching a table with
httr::POST, the query should be placed in the
body.
Note: at the end of the query you may change
"format": "px" to "format": "json" in order to
get a reply in JSON-format. Fetch a table, examine its structure and try
to extract a suitable data.frame. See SCB_KPI_API.R
for a simple example.
Note: A dedicated R package pxweb
exists for querying the SCB database through the API in a slightly more
comfortable way. See the vignette
for a demonstratio.
The last days hourly temperatures (parameter 1) at Bromma (station
97200) can be fetched from SMHI by (switch xml for
json if you want to change format)
temp_response <- GET("https://opendata-download-metobs.smhi.se/api/version/1.0/parameter/1/station/97200/period/latest-day/data.xml")
http_type(temp_response)## [1] "application/xml"We extract the XML-content by
library(xml2)
temp_xml <- read_xml(temp_response)
class(temp_xml)## [1] "xml_document" "xml_node"The structure can be viewed by opening
in your web-browser. We see that temperatures can be found in
XPATH "/metObsSampleData/value/value":
xml_ns_strip(temp_xml) # Överkurs
xml_find_all(temp_xml, "/metObsSampleData/value/value")## {xml_nodeset (25)}
## [1] <value>1.9</value>
## [2] <value>1.9</value>
## [3] <value>1.6</value>
## [4] <value>1.4</value>
## [5] <value>1.1</value>
## [6] <value>0.9</value>
## [7] <value>0.7</value>
## [8] <value>0.7</value>
## [9] <value>0.7</value>
## [10] <value>0.5</value>
## [11] <value>0.4</value>
## [12] <value>0.3</value>
## [13] <value>0.0</value>
## [14] <value>-0.2</value>
## [15] <value>-0.4</value>
## [16] <value>-0.7</value>
## [17] <value>-1.0</value>
## [18] <value>-1.4</value>
## [19] <value>-1.6</value>
## [20] <value>-1.6</value>
## ...(use xml_text to get the values).