Command line interface#
The showyourwork! package implements a single command-line utility:
showyourwork
, which allows users to set up, configure, and build their
open source article. Below we describe this command and discuss its various
subcommands.
showyourwork
#
showyourwork --help
Usage: showyourwork [OPTIONS] COMMAND [ARGS]...
Easily build open-source, reproducible scientific articles.
Options:
-v, --version Show the version and exit.
--help Show this message and exit.
Commands:
build Build an article in the current working directory.
cache Caching-related operations.
clean Clean the article build in the current working directory.
setup Set up a new article repository in the current working directory.
tarball Generate a tarball of the build in the current working directory.
Running showyourwork! (without any arguments) is a shortcut for showyourwork build
(see showyourwork build below).
showyourwork setup
#
showyourwork setup --help
Usage: showyourwork setup [OPTIONS] <user/repo>
Set up a new article repository in the current working directory.
This command expects a single positional argument, `SLUG`, of the form
`user/repo`, where `user` is the user's GitHub handle and `repo` is the name
of the repository (and local directory) to create.
Options:
-y, --yes Accept all `Press any key to continue` prompts
automatically.
-q, --quiet Don't prompt the user, and don't display
informational output.
-c, --cache Set up intermediate result caching on Zenodo
Sandbox. Requires a Zenodo Sandbox API token
provided as the environment variable and GitHub
repository secret `SANDBOX_TOKEN`.
-o, --overleaf PROJECT_ID Overleaf project id to sync with (optional).
Requires Overleaf credentials, provided as the
environment variables and GitHub repository
secrets `OVERLEAF_EMAIL` and `OVERLEAF_PASSWORD`.
-s, --ssh Use ssh to authenticate with GitHub? Default is
to use https.
-a, --action-spec SPEC Version spec of showyourwork to use in the GitHub
action. Default is the latest version on PyPI.
--help Show this message and exit.
The setup
subcommand sets up an open source article repository from scratch
in the current working directory. This is an interactive command (unless you
provide the --yes
or --quiet
options; see below).
Let’s step through what it does here.
To set up a new open source article repository, run
showyourwork setup $USER/$REPO
where you should replace $USER
with your GitHub user name and
$REPO
with the name of your new article repository. For definiteness,
here we’ll use my user name (rodluger
) and we’ll call our repository
article
.
Step 1#
Running the setup
command as above should bring up the following prompt:
Let's get you set up with a new repository. I'm going to create a folder called article in the current working directory. If you haven't done this yet, please visit https://github.com/new at this time and create an empty repository called rodluger/article
As requested, if you haven’t yet created the remote repository, go to github.com/new in your browser to create an empty repository of the same name. There’s no need to create a README, gitignore file, or LICENSE at this time, as showyourwork! will set those up for you.
Press any key to bring up the next prompt. What you see next depends on whether
or not you provided the --cache
option.
Step 2A#
If you didn’t request Zenodo caching functionality (see below), you’ll see the following the message:
You didn't provide a caching service (via the --cache
command-line option), so I'm not going to set up remote caching for this repository.
Step 2B#
If instead you passed the --cache
flag, you’ll see the following message:
You requested remote caching on Zenodo, so I'm going to create a deposit draft where intermediate results will be cached. Please make sure at this time that you have defined the ZENODO_TOKEN environment variable containing your API key for Zenodo. If you don't have one, visit https://zenodo.org/account/settings/applications/tokens/new to create a new personal access token with deposit:actions and deposit:write scopes and store it in the environment variable ZENODO_TOKEN. In order for this to work on GitHub Actions, you'll also have to visit https://github.com/tmp/tmp/settings/secrets/actions/new at this time to create a ZENODO_TOKEN secret with your API access token.
Note that, in addition to the --cache
flag, which enables caching on Zenodo,
users may also provide the --sandbox
flag, which switches the host to Zenodo
Sandbox. Zenodo Sandbox behaves in exactly the same way as Zenodo, but it is
explicitly meant as a test bed for dataset archiving. While deposits on Sandbox get assigned
DOIs, they are no actual registered DOIs and have a limited lifespan.
Sandbox is therefore a great choice for debugging and development; read more about
it at Zenodo integration. Note that if you choose the --sandbox
option, you’ll need
a Zenodo Sandbox API token stored int the SANDBOX_TOKEN
environment variable
and GitHub Actions secret.
Warning
Never commit your Zenodo API token (or any API token) directly to your repository!
You can read more about GitHub secrets (and the security measures in place to prevent them from getting exposed to the outside world) at the GitHub documentation.
Press any key to bring up the next prompt. What you see next depends on whether
or not you specified the --overleaf
option.
Step 3A#
If you didn’t pass the --overleaf
option, you’ll see the following:
You didn't provide an Overleaf project id (via the --overleaf command-line
option), so I'm not going to set up Overleaf integration for this repository.
If you would like to set up integration with an Overleaf project (see Overleaf integration),
hit Ctrl+C
and run
showyourwork setup --overleaf=62150dd16134ef045f81d1c8
where you should replace 62150dd16134ef045f81d1c8
with the 24-character id
of a new (blank) Overleaf project. Once you create a new Overleaf project, you
can grab the id from the last bit of the project’s URL. Note that showyourwork!
requires the Overleaf project to be empty, otherwise it will refuse to set up
the integration. For more information on how this integration works, and what
to do if you have an existing Overleaf project you’d like to integrate with
showyourwork!, please see Overleaf integration.
Step 3B#
If you specified the --overleaf
option (see Step 3A),
you’ll get the following message:
You provided an Overleaf project id, so I'm going to set up Overleaf integration for this repository. Please make sure at this time that you have defined the OVERLEAF_EMAIL and OVERLEAF_PASSWORD environment variables. In order for this to work on GitHub Actions, please go to https://github.com/tmp/tmp/settings/secrets/actions/new at this time and create OVERLEAF_EMAIL and OVERLEAF_PASSWORD secrets with your Overleaf credentials.
To allow showyourwork! to push to/pull from your Overleaf project, create
the environment variables $OVERLEAF_EMAIL
and $OVERLEAF_PASSWORD
and
populate them with your Overleaf email address and password, respectively;
then re-run the setup command.
Again, take care to never actually commit this information to your repository!
Step 4#
Finally, press any key to generate the repository. This will create a new folder
in the current working directory with the same name as your repo (article
, in
the example above) and set up git
tracking for it. Note that the first time
you commit and push your changes to the GitHub repository, you’ll have to specify
the upstream branch as follows:
git push -u origin main
showyourwork build
#
showyourwork build --help
Usage: showyourwork build [OPTIONS] [SNAKEMAKE_ARGS]...
Build an article in the current working directory.
Options:
-c, --cores TEXT Number of cores to use; passed to snakemake.
--conda-frontend TEXT The conda frontend to use; passed to snakemake.
-p, --project TEXT The root folder of the showyourwork project.
--help Show this message and exit.
Run this command to build the article in the current working directory. Note that
you must run this command from the top level of the repository (an error will
be thrown otherwise). The command accepts any number of arguments, all of which
are forwarded to snakemake
.
By default, showyourwork
passes the following arguments to snakemake
:
-c1 --use-conda --reason --cache
Some of these, like the number of cores, can be overridden. For example, you may run
showyourwork build -c2
to run the workflow using two cores (see the snakemake docs
for details). Additional arguments can also be provided, like --verbose
to increase
the verbosity of the Snakemake logs (see Logging), or --force
and --forceall
to
force the re-execution of the rule that builds the manuscript or all of the rules
in the workflow, respectively (regardless of whether the outputs are up to date
or not). Positional arguments are also allowed; for instance, to only build a specific
figure, you may run, e.g.,
showyourwork build --force src/tex/figures/figure.pdf
You can check out the complete list of Snakemake arguments and options at the snakemake documentation.
Warning
Not all Snakemake options are compatible with showyourwork!. If you run into issues when specifying custom options, please let us know.
Note that the build process in showyourwork! happens in two steps, each of
which executes a separate Snakemake workflow. The first
step is a preprocessing step that parses the user config file and does a quick
first-pass compiling of the TeX manuscript to look for \includegraphics
and \script
calls, which it uses to build the graph of dependencies for
your article. The second step is the main step, in which all of the dependencies
are built (if needed) and the final article PDF is generated. Arguments
passed to showyourwork build
are ingested only during the second step.
Finally, showyourwork! takes full advantage
of the dependency tracking and caching functionality of Snakemake. When
running showyourwork build
, only files whose upstream dependencies have
changed (since the last build) will be re-generated. This is true even when
running on GitHub Actions; the showyourwork-action
caches results across
runs to minimize compute time for the build. We even go a step further, and
extend the Snakemake functionality to allow caching of intermediate
dependencies on Zenodo; read about it at Zenodo integration.
showyourwork clean
#
showyourwork clean --help
Usage: showyourwork clean [OPTIONS] [SNAKEMAKE_ARGS]...
Clean the article build in the current working directory.
Options:
-c, --cores TEXT Number of cores to use; passed to snakemake.
--conda-frontend TEXT The conda frontend to use; passed to snakemake.
-p, --project TEXT The root folder of the showyourwork project.
-f, --force Forcefully remove everything in the `src/tex/figures`
and `src/data` directories.
-d, --deep Forcefully remove the `.snakemake` and
`~/.showyourwork` directories.
--help Show this message and exit.
This command removes all of the output from previous showyourwork build
steps. Depending on the state of your repository, and if there are errors in
your config file or missing dependencies, this command may fail silently, in
which case some of the output may remain after running it.
Force clean#
If showyourwork clean
didn’t remove all of the output, you can force
the deletion of all the programmatically-generated figures and datasets by
passing the --force
option, which will remove
everything in the src/tex/figures
, src/data
, and temporary
.showyourwork
folders
(assuming you’re respecting the showyourwork! conventions; see Repository layout).
Deep clean#
If you want to start over from scratch, you can also pass the --deep
option
for a deep clean. This removes
the hidden .snakemake
folder, which houses the conda
environments
for your build (among other things), so
deleting it will force a re-install of all packages used in your workflow.
This option also removes the .showyourwork
folder located in your $HOME
path, which also houses conda
environments
used at different stages of the build step. You can safely remove it at any time
(at the cost of a longer runtime the next time you execute showyourwork
).
showyourwork tarball
#
showyourwork tarball --help
Usage: showyourwork tarball [OPTIONS] [SNAKEMAKE_ARGS]...
Generate a tarball of the build in the current working directory.
Options:
-c, --cores TEXT Number of cores to use; passed to snakemake.
--conda-frontend TEXT The conda frontend to use; passed to snakemake.
-p, --project TEXT The root folder of the showyourwork project.
--help Show this message and exit.
Like build
, the showyourwork tarball
command builds your article, but
also gathers all of the relevant files needed to build it using a standard
TeX engine into a tarball called arxiv.tar.gz
. It’s named that because
you should be able to directly upload this tarball when submitting a paper
to the arXiv article service.
showyourwork cache
#
showyourwork cache --help
Usage: showyourwork cache [OPTIONS] COMMAND [ARGS]...
Caching-related operations.
Options:
--help Show this message and exit.
Commands:
create Create a Zenodo Sandbox deposit draft for the given branch.
delete Delete the latest draft of a Zenodo Sandbox deposit.
freeze Publishes the current Zenodo Sandbox deposit draft for the...
publish Publishes the current Zenodo Sandbox deposit draft for the...
Subcommand documentation:
Usage: showyourwork cache create [OPTIONS]
Create a Zenodo Sandbox deposit draft for the given branch.
Requires a Zenodo Sandbox API token provided as the environment variable and
Github repository secret `SANDBOX_TOKEN`.
Options:
-b, --branch TEXT Branch whose deposit is to be deleted. Default is current
branch.
--help Show this message and exit.
Usage: showyourwork cache delete [OPTIONS]
Delete the latest draft of a Zenodo Sandbox deposit.
Requires a Zenodo Sandbox API token provided as the environment variable and
Github repository secret `SANDBOX_TOKEN`.
Options:
-b, --branch TEXT Branch whose deposit is to be deleted. Default is current
branch.
--help Show this message and exit.
Usage: showyourwork cache freeze [OPTIONS]
Publishes the current Zenodo Sandbox deposit draft for the given branch to
Zenodo Sandbox.
Useful for preserving the current cache state even when changes are made to
the rule inputs. Note that Zenodo Sandbox storage is temporary, so if you'd
like to publish your cache on Zenodo, you should call ``publish`` instead.
Requires a Zenodo Sandbox API token provided as the environment variable and
Github repository secret `SANDBOX_TOKEN`.
Options:
-b, --branch TEXT Branch whose deposit is to be deleted. Default is current
branch.
--help Show this message and exit.
Usage: showyourwork cache publish [OPTIONS]
Publishes the current Zenodo Sandbox deposit draft for the given branch to
Zenodo for permanent, public storage. This action assigns a permanent,
static DOI to the deposit, which cannot be deleted or undone.
Requires a Zenodo API token provided as the environment variable
`ZENODO_TOKEN`.
Options:
-b, --branch TEXT Branch whose deposit is to be deleted. Default is current
branch.
--help Show this message and exit.
Utilities for creating, deleting, and publishing the Zenodo deposit drafts used to cache intermediate results from your workflow; see Zenodo integration.