class: center, middle, inverse, title-slide # Automatically Update your Report with Github Actions ## Teaching Demonstration for UCSB Masters in Env. Data Science ###
Ben Best (PhD)
•
@ben_d_best
###
EcoQuants.com
### 2021-05-04 --- class: top, left ## May the Fourth be with You! <img src="./images/starwars-rey-rstats_sz-m.png" width="60%" style="display: block; margin: auto;" /> Credit: Artwork from @juliesquid for @openscapes (illustrated by @allison_horst). ??? - Presentation shortcuts: - `c`: clone synchronized window - `p`: presenter notes - Star Wars figure: - Ok, so it's May 4th -- couldn't resist "Star Wars Day" reference - empower students with tools - work with community, be inclusive --- ## Course ### EDS 214: Analytical Workflows and Scientific Reproducibility _2 units, offered in Summer_ The generation and analysis of environmental data is often a complex, multi-step process that may involves the <mark>collaboration</mark> of many people. Increasingly tools that document help to organize and document <mark>workflows</mark> are being used to ensure <mark>reproducibility</mark> and transparency of the results. This course will introduce students to conceptual organization of workflows as a way to conduct reproducible analyses, as well as various software tools that help users to manage multi-step processes that requires tools for storing, managing and <mark>sharing</mark> workflows, code, documents and data, including <mark>GitHub</mark>, Pegasus, remake and Kepler. --- ## Example ### Solar in Cali <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:orange;" xmlns="http://www.w3.org/2000/svg"> <path d="M256 160c-52.9 0-96 43.1-96 96s43.1 96 96 96 96-43.1 96-96-43.1-96-96-96zm246.4 80.5l-94.7-47.3 33.5-100.4c4.5-13.6-8.4-26.5-21.9-21.9l-100.4 33.5-47.4-94.8c-6.4-12.8-24.6-12.8-31 0l-47.3 94.7L92.7 70.8c-13.6-4.5-26.5 8.4-21.9 21.9l33.5 100.4-94.7 47.4c-12.8 6.4-12.8 24.6 0 31l94.7 47.3-33.5 100.5c-4.5 13.6 8.4 26.5 21.9 21.9l100.4-33.5 47.3 94.7c6.4 12.8 24.6 12.8 31 0l47.3-94.7 100.4 33.5c13.6 4.5 26.5-8.4 21.9-21.9l-33.5-100.4 94.7-47.3c13-6.5 13-24.7.2-31.1zm-155.9 106c-49.9 49.9-131.1 49.9-181 0-49.9-49.9-49.9-131.1 0-181 49.9-49.9 131.1-49.9 181 0 49.9 49.9 49.9 131.1 0 181z"></path></svg>
Source: [U.S. Energy Information Administration (EIA)](https://www.eia.gov/opendata/qb.php?category=3389873&sdid=EBA.CAL-ALL.NG.SUN.H). ??? - Within last week --- ## Example ### R Code <svg viewBox="0 0 640 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:grey;" xmlns="http://www.w3.org/2000/svg"> <path d="M278.9 511.5l-61-17.7c-6.4-1.8-10-8.5-8.2-14.9L346.2 8.7c1.8-6.4 8.5-10 14.9-8.2l61 17.7c6.4 1.8 10 8.5 8.2 14.9L293.8 503.3c-1.9 6.4-8.5 10.1-14.9 8.2zm-114-112.2l43.5-46.4c4.6-4.9 4.3-12.7-.8-17.2L117 256l90.6-79.7c5.1-4.5 5.5-12.3.8-17.2l-43.5-46.4c-4.5-4.8-12.1-5.1-17-.5L3.8 247.2c-5.1 4.7-5.1 12.8 0 17.5l144.1 135.1c4.9 4.6 12.5 4.4 17-.5zm327.2.6l144.1-135.1c5.1-4.7 5.1-12.8 0-17.5L492.1 112.1c-4.8-4.5-12.4-4.3-17 .5L431.6 159c-4.6 4.9-4.3 12.7.8 17.2L523 256l-90.6 79.7c-5.1 4.5-5.5 12.3-.8 17.2l43.5 46.4c4.5 4.9 12.1 5.1 17 .6z"></path></svg> ```r # setup variables api_key <- "5deaec228369f8806a73a6684f89cc74" series <- list( id = "EBA.CAL-ALL.NG.SUN.H", name = "Net generation from solar for California, hourly", units = "MWh") # get data x <- EIAdata::getEIA(series$id, api_key) tzone(x) <- Sys.getenv("TZ") y <- xts::last(x, "7 days") # plot data dygraphs::dygraph(y, ylab = series$units, main = series$name) ``` -- - <svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> code: [github.com/bbest/gh-action-test-rmd](https://github.com/bbest/gh-action-test-rmd) - <svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M336.5 160C322 70.7 287.8 8 248 8s-74 62.7-88.5 152h177zM152 256c0 22.2 1.2 43.5 3.3 64h185.3c2.1-20.5 3.3-41.8 3.3-64s-1.2-43.5-3.3-64H155.3c-2.1 20.5-3.3 41.8-3.3 64zm324.7-96c-28.6-67.9-86.5-120.4-158-141.6 24.4 33.8 41.2 84.7 50 141.6h108zM177.2 18.4C105.8 39.6 47.8 92.1 19.3 160h108c8.7-56.9 25.5-107.8 49.9-141.6zM487.4 192H372.7c2.1 21 3.3 42.5 3.3 64s-1.2 43-3.3 64h114.6c5.5-20.5 8.6-41.8 8.6-64s-3.1-43.5-8.5-64zM120 256c0-21.5 1.2-43 3.3-64H8.6C3.2 212.5 0 233.8 0 256s3.2 43.5 8.6 64h114.6c-2-21-3.2-42.5-3.2-64zm39.5 96c14.5 89.3 48.7 152 88.5 152s74-62.7 88.5-152h-177zm159.3 141.6c71.4-21.2 129.4-73.7 158-141.6h-108c-8.8 56.9-25.6 107.8-50 141.6zM19.3 352c28.6 67.9 86.5 120.4 158 141.6-24.4-33.8-41.2-84.7-50-141.6h-108z"></path></svg> website: [bbest.github.io/gh-action-test-rmd](https://bbest.github.io/gh-action-test-rmd/) --- ## Motivation ### How to make this report '_**Evergreen**_'? <svg viewBox="0 0 384 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:green;" xmlns="http://www.w3.org/2000/svg"> <path d="M378.31 378.49L298.42 288h30.63c9.01 0 16.98-5 20.78-13.06 3.8-8.04 2.55-17.26-3.28-24.05L268.42 160h28.89c9.1 0 17.3-5.35 20.86-13.61 3.52-8.13 1.86-17.59-4.24-24.08L203.66 4.83c-6.03-6.45-17.28-6.45-23.32 0L70.06 122.31c-6.1 6.49-7.75 15.95-4.24 24.08C69.38 154.65 77.59 160 86.69 160h28.89l-78.14 90.91c-5.81 6.78-7.06 15.99-3.27 24.04C37.97 283 45.93 288 54.95 288h30.63L5.69 378.49c-6 6.79-7.36 16.09-3.56 24.26 3.75 8.05 12 13.25 21.01 13.25H160v24.45l-30.29 48.4c-5.32 10.64 2.42 23.16 14.31 23.16h95.96c11.89 0 19.63-12.52 14.31-23.16L224 440.45V416h136.86c9.01 0 17.26-5.2 21.01-13.25 3.8-8.17 2.44-17.47-3.56-24.26z"></path></svg> Normally on your personal computer (**PC**) <svg viewBox="0 0 640 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M624 416H381.54c-.74 19.81-14.71 32-32.74 32H288c-18.69 0-33.02-17.47-32.77-32H16c-8.8 0-16 7.2-16 16v16c0 35.2 28.8 64 64 64h512c35.2 0 64-28.8 64-64v-16c0-8.8-7.2-16-16-16zM576 48c0-26.4-21.6-48-48-48H112C85.6 0 64 21.6 64 48v336h512V48zm-64 272H128V64h384v256z"></path></svg>: - Open RStudio -- - Knit Rmarkdown file to html -- - Git commit & push to Github, where website hosted -- - Manually run, every month/week/day/hour ?! 😭 --
--- ## Motivation ### How to make this report '_**Evergreen**_'? <svg viewBox="0 0 384 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:green;" xmlns="http://www.w3.org/2000/svg"> <path d="M378.31 378.49L298.42 288h30.63c9.01 0 16.98-5 20.78-13.06 3.8-8.04 2.55-17.26-3.28-24.05L268.42 160h28.89c9.1 0 17.3-5.35 20.86-13.61 3.52-8.13 1.86-17.59-4.24-24.08L203.66 4.83c-6.03-6.45-17.28-6.45-23.32 0L70.06 122.31c-6.1 6.49-7.75 15.95-4.24 24.08C69.38 154.65 77.59 160 86.69 160h28.89l-78.14 90.91c-5.81 6.78-7.06 15.99-3.27 24.04C37.97 283 45.93 288 54.95 288h30.63L5.69 378.49c-6 6.79-7.36 16.09-3.56 24.26 3.75 8.05 12 13.25 21.01 13.25H160v24.45l-30.29 48.4c-5.32 10.64 2.42 23.16 14.31 23.16h95.96c11.89 0 19.63-12.52 14.31-23.16L224 440.45V416h136.86c9.01 0 17.26-5.2 21.01-13.25 3.8-8.17 2.44-17.47-3.56-24.26z"></path></svg> Entrez **Github Actions** <svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> (no PC needed)
-- Use Github Action to render the Rmarkdown: ```r usethis::use_github_action("render-rmarkdown") ``` -- ``` ✓ Setting active project to '/Users/bbest/github/bbest/gh-action-test-rmd' ✓ Creating '.github/' ✓ Adding '^\\.github$' to '.Rbuildignore' ✓ Adding '*.html' to '.github/.gitignore' ✓ Creating '.github/workflows/' ✓ Writing '.github/workflows/render-rmarkdown.yaml' ● Learn more at <https://github.com/r-lib/actions/blob/master/examples/README.md> ``` --- ## Yet another Markup Language (YAML) ### Initial `render-rmarkdown.yaml`, `on.push` `.github/workflows/`[`render-rmarkdown.yaml`](https://github.com/bbest/gh-action-test-rmd/blob/670e68f67ed2f768144702746b4b247b1144fb09/.github/workflows/render-rmarkdown.yaml) (commit: `670e68f`) ```yaml on: push: paths: - '**.Rmd' ``` Only updates when changes are sent with `git push`. --- ## Yet another Markup Language (YAML) ### Initial `render-rmarkdown.yaml`, `jobs.build` `.github/workflows/`[`render-rmarkdown.yaml`](https://github.com/bbest/gh-action-test-rmd/blob/670e68f67ed2f768144702746b4b247b1144fb09/.github/workflows/render-rmarkdown.yaml) (commit: `670e68f`) ```yaml jobs: build: runs-on: macOS-latest env: GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} ... ``` Uses "secrets" already in Github to update the repository of code. --- ## Yet another Markup Language (YAML) ### Modify `render-rmarkdown.yaml`, `on.push|schedule` `.github/workflows/`[`render-rmarkdown.yaml`](https://github.com/bbest/gh-action-test-rmd/blob/main/.github/workflows/render-rmarkdown.yaml) (commit: latest) ```yaml on: push: schedule: - cron: '0 * * * *' ``` -- - `push`: now without restriction for changes to `*.Rmd` files -- - `schedule.cron`: See [Events that trigger workflows - GitHub Actions Docs](https://docs.github.com/en/actions/reference/events-that-trigger-workflows#schedule) -- - **Question**: What's the cron syntax to run every Monday? Every 8th day? --- ## Yet another Markup Language (YAML) ### Modify `render-rmarkdown.yaml`, `on.workflow_dispatch` `.github/workflows/`[`render-rmarkdown.yaml`](https://github.com/bbest/gh-action-test-rmd/blob/main/.github/workflows/render-rmarkdown.yaml) (commit: latest) ```yaml on: push: schedule: - cron: '0 * * * *' workflow_dispatch: ``` -- - `workflow_dispatch`: get [Actions button](https://github.com/bbest/gh-action-test-rmd/actions/workflows/render-rmarkdown.yaml) to run through Github web interface ![](./images/gh-action_workflow-dispatch.png) --- ## Github Actions Summary With Github Actions, you can: - Spin up a dedicated machine to crunch data and update your reports! -- - For FREE! -- - Within limits: ~ 1 hr with ~ 1 GB and small CPU --- ## More Actions... - `usethis::`[`use_github_action()`](https://usethis.r-lib.org/reference/use_github_action.html) - [r-lib/actions: examples/](https://github.com/r-lib/actions/tree/master/examples#example-workflows) -- RMarkdown workflows: - [`render-rmarkdown`](https://github.com/r-lib/actions/tree/master/examples#render-rmarkdown) - Render one or more Rmarkdown files when they change and commit the result. - [`bookdown`](https://github.com/r-lib/actions/tree/master/examples#build-bookdown-site) - Build a [bookdown] site and deploy it to [netlify]. - [`blogdown`](https://github.com/r-lib/actions/tree/master/examples#build-blogdown-site) - Build a [blogdown] site and deploy it to [netlify]. --- class: center, middle, inverse # Thanks! <style type="text/css"> h2 { text-align: center } </style> ## Questions / Comments ? Example<br> [github.com/bbest/gh-action-test-rmd](https://github.com/bbest/gh-action-test-rmd)<br> ([starlog](http://starlogs.net/#bbest/gh-action-test-rmd) ✨) .bottom[.right[.small[ Slides created via the R package [**xaringan**](https://github.com/yihui/xaringan) and [R Markdown](https://rmarkdown.rstudio.com). ]]] [GitHub Pages]: https://pages.github.com/ [renv]: https://rstudio.github.io/renv/ [pkgdown]: https://pkgdown.r-lib.org/ [bookdown]: https://bookdown.org [blogdown]: https://bookdown.org/yihui/blogdown/ [netlify]: https://www.netlify.com/