Github actions for automated testing
A beginner's guide
I recently started using github actions to test my python package sktools. It looks like a fairly easy tool to use, and I find it very powerful.
The idea is to create workflows that automate actions for you. According to the docs:
Workflows are custom automated processes that you can set up in your repository to build, test, package, release, or deploy any project on GitHub.
Creating a workflow is as simple as creating a yml file in .github/workflows
. The file I’ve created looks like this, and it installs the sktools package in several environments and runs its tests:
# Action title
name: Unit Tests
# Controls when the action will run.
on:
schedule:
- cron: "0 0 * * *"
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: $
strategy:
matrix:
python-version: [3.6, 3.7]
os: [macos-10.15, ubuntu-latest, windows-latest]
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Runs a single command using the runners shell
- uses: actions/checkout@v2
- name: Set up Python $
uses: actions/setup-python@v1
with:
python-version: $
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install sktools
pip freeze
- name: Test with pytest
run: |
make test
Let’s look at what each step does.
The name
key is provided so it’ll be easily identified in the actions tab. The on
key controls when the workflow will be run. In this case, it is scheduled via a cron that runs the action on a daily basis, at midnight.
# Workflow title
name: Unit Tests
# Controls when the action will run.
on:
schedule:
- cron: "0 0 * * *"
Alternative on
options are:
on:
push:
branches:
- master
pull_request:
branches:
- master
Which would run the workflow when master branch is updated and on pull requests. Of course on
can handle all of them at the same time.
After that, you can specify several jobs for the workflow to run. In our case, we’ve specified a single job called build
. We’ve also specified on which environments do we need to run it on. The idea is that the instructions that we’ll specify afterwards will run in mac, ubuntu and windows and both in python 3.6 and 3.7 for each of the OS. And this is without even having python 3.6 in my computer, and only one OS. This way I can ensure that my library works not only in my computer.
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: $
strategy:
matrix:
python-version: [3.6, 3.7]
os: [macos-10.15, ubuntu-latest, windows-latest]
In the following lines we define the steps that each execution of the workflow will carry. Each step is defined by a yaml key, the first one being the checkout. The checkout gets the latest commit of your repo. The second action, with name Set up Python 3.x
, will create a python environment.
Both these steps are open source (setup python, checkout), and you can also build your own custom steps.
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Runs a single command using the runners shell
- uses: actions/checkout@v2
- name: Set up Python $
uses: actions/setup-python@v1
with:
python-version: $
The last steps are the most specific to our job. The “Install dependencies” job installs sktools and the last step runs the tests in sktools.
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install sktools
pip freeze
- name: Test with pytest
run: |
make test
The make test
is specific to our project as we run the tests from a Makefile. However, as in the Makefile we have the lines
test:
python setup.py test
changing the make test
by python setup.py test
won’t make any difference. By looking in the workflow run in the actions tab on github, we can see in its logs:
test_basic_featurizer (tests.test_sktools.TestGroupQuantileFeaturizer) ... ok
test_missing (tests.test_sktools.TestGroupQuantileFeaturizer) ... ok
test_new_input (tests.test_sktools.TestGroupQuantileFeaturizer) ... ok
test_select_items (tests.test_sktools.TestItemSelector) ... ok
test_zero_matrix (tests.test_sktools.TestMatrixDenser) ... ok
test_basic_featurizer (tests.test_sktools.TestMeanFeaturizer) ... ok
test_missing (tests.test_sktools.TestMeanFeaturizer) ... ok
test_new_input (tests.test_sktools.TestMeanFeaturizer) ... ok
test_float_works (tests.test_sktools.TestTypeSelector) ... ok
test_integer_works (tests.test_sktools.TestTypeSelector) ... ok
test_object_works (tests.test_sktools.TestTypeSelector) ... ok
test_all_missing (tests.test_encoders.TestNestedTargetEncoder) ... ok
test_missing_na (tests.test_encoders.TestNestedTargetEncoder) ... ok
test_no_parent (tests.test_encoders.TestNestedTargetEncoder) ... ok
test_numpy_array (tests.test_encoders.TestNestedTargetEncoder) ... ok
test_parent_prior (tests.test_encoders.TestNestedTargetEncoder) ... ok
test_unknown_missing_imputation (tests.test_encoders.TestNestedTargetEncoder) ... ok
test_max_works (tests.test_encoders.TestQuantileEncoder) ... ok
test_median_works (tests.test_encoders.TestQuantileEncoder) ... ok
test_new_category (tests.test_encoders.TestQuantileEncoder) ... ok
test_several_quantiles (tests.test_encoders.TestSummaryEncoder) ... ok
test_period_mapping (tests.test_preprocessing.TestCyclicFeaturizer)
Expect same output by specifying period mapping ... ok
test_trigonometry (tests.test_preprocessing.TestCyclicFeaturizer)
Expect cosines and sines to work ... ok
test_with_intercept (tests.test_linear_model.TestQuantileRegression) ... ok
test_without_intercept (tests.test_linear_model.TestQuantileRegression) ... ok
test_1_tree (tests.test_ensemble.TestMedianForest) ... ok
test_2_trees (tests.test_ensemble.TestMedianForest) ... ok
test_many_trees (tests.test_ensemble.TestMedianForest) ... ok
test_cross_val_integration (tests.test_model_selection.TestBootstrapFold)
Check cv is compatible with cross_val_score ... ok
test_grid_integration (tests.test_model_selection.TestBootstrapFold)
Check that cv is compatible with GridSearchCV ... ok
test_n_splits (tests.test_model_selection.TestBootstrapFold)
Check that get_n_splits returns the number of bootstraps ... ok
test_size_fraction_works (tests.test_model_selection.TestBootstrapFold)
Check that size_fraction works as expected ... ok
----------------------------------------------------------------------
Ran 36 tests in 2.041s
OK
The tests are passing, yay!