Wrglctl reference
Wrglctl is our official command-line client. It is meant for CI use cases but you can also use it as a convenient way to interact with WRGL from the command line. It allows you to create commit, download CSV, show commit logs and more.
Download
Download & install instruction for each OS.
Environment variables
When using Wrglctl in CI it's easier to login using these environment variables. Note that you need to set both WRGL_APIKEY
and WRGL_USERNAME
for operations to work smoothly.
Env var | Description |
---|---|
WRGL_APIKEY | The API key to authenticate your requests. |
WRGL_USERNAME | Your username. |
Login
You can login with email/password or an API key. In either case an API key will be saved in the machine for subsequent runs.
Usage
wrglctl login [flags]
Flags
Full | Short | Description |
---|---|---|
--apikey |
-k |
API key string. |
Create commit
Commit a CSV file and possibly create the repo if it does not already exist.
Usage
wrglctl repos commit CSV_FILE_PATH REPO_NAME MESSAGE [flags]
Arguments
Arguments | Description |
---|---|
CSV_FILE_PATH | Path to the CSV file. |
REPO_NAME | Repository name. If the repo already exist then append this commit, otherwise create it. |
MESSAGE | Commit's purpose in plain English. |
Flags
Full | Short | Description |
---|---|---|
--primary-key |
-p |
Specify which columns are primary key. Multiple columns can be separated by comma. e.g. "pid,year". If primary key is not specified then all columns are included in the primary key. |
--validate-row-uniqueness |
-i |
Print an error if there are rows with duplicated primary key. |
List repositories
Show all repositories.
Usage
wrglctl repos list [flags]
Flags
Full | Short | Description |
---|---|---|
--no-pager | -P | Don't use pager (specified by env var PAGER ) to paginate output. Print everything at once. |
Show commit logs
Show commit logs for a repo. Output very similar to git log
.
Usage
wrglctl repos log REPO_NAME [flags]
Arguments
Arguments | Description |
---|---|
REPO_NAME | Name of repository to show logs. |
Flags
Full | Short | Description |
---|---|---|
--no-pager | -P | Don't use pager (specified by env var PAGER ) to paginate output. Print everything at once. |
Pull CSV
Download the latest commit from a repo and save to a local CSV file. You can also pull an earlier commit by specifying a hash (can be looked up with wrglctl repos log
).
Usage
wrglctl repos pull REPO_NAME OUTPUT_FILE [flags]
Arguments
Arguments | Description |
---|---|
REPO_NAME | Name of repository to pull from. |
OUTPUT_FILE | Save CSV to this file. |
Flags
Full | Short | Description |
---|---|---|
--commithash |
-H |
Specify the exact commit to pull. |
Sync multiple repos
When you work with multiple repositories it is easy to lose track of which file commit to which repo, which repos are up-to-date, etc. You can write a script to automate some of that but it still isn't easy to get right. repos sync
command will help with that by keeping track of multiple repos and only pull newer files. And when you make changes to those files locally, it look at file's mod time and will only commit the files that you updated. Furthermore a sync file can be linked to a parent Git repo from which it will read Git commit info and put that in the WRGL commit message.
This command generate 2 files in sync root folder:
.wrglsync.yaml
: This file is called "sync file". It configures behavior of sync command. You will want to commit this file in your Git repo..wrglmod.yaml
: This file is called "mod file" and it keep track of modification time for each file so that only updated files are committed during sync. You will want to add this file to.gitignore
as to not commit it to Git repo.
Usage
wrglctl repos sync [DIRECTORY] [flags]
Arguments
Arguments | Description |
---|---|
DIRECTORY | Optional. This is the sync root directory. If not specified then it is current working directory. |
Sync file
When you run repos sync
for the first time. You will be asked a series of questions to create a sync file with the name .wrglsync.yaml
in a folder of your choosing which will also be the root folder for files that need to be synced.
Fields | Description |
---|---|
username | The username that the repos belong to. This field become relevant when team feature is introduced. |
repos | A map of repos managed by sync command. Each key is the repo name while the value is a repo sync object. |
commit | Optional, only add when you wish for repos sync to commit files for you as well. |
commit.messageTemplate | This is a template for commit message written in Go template language. For now its only use case is to add info from Git commit to your WRGL commit message. Therefore only the following keys are available:
|
commit.gitRoot | The Git repo directory from which Git info will be read (and linked to WRGL commit), must be one of the parent folders to the sync root folder. If you create sync file with repos sync then it will help you select the correct folder. |
commit.gitRemote | This field tell repos sync command how to construct GitCommit.Link during commit. The recommended way to populate this field is to let repos sync command fill it as part of sync file creation. It should work with major Git hosts such as GitHub, GitLab, BitBucket, Azure, ... Though for now only GitHub is well-tested. |
Repo sync object
Fields | Description |
---|---|
fileName | Relative path to the file that will be committed/updated for this repo. |
primaryKey | Optional, this is the list of columns included in primary key when repo is committed. |
Create sync file in current folder if none exist:
wrglctl repos sync
Example content of sync file:
username: ipno
repos:
personel:
fileName: fuse/personel.csv
primaryKey:
- uid
personel_history:
fileName: fuse/personel_history.csv
complaint:
fileName: fuse/complaint.csv
commit:
messageTemplate: |-
{{.GitCommit.Link}}
{{.GitCommit.Message}}
gitRemote:
host: github.com
dir: ppact/processing
gitRoot: ..
Pull newer files and commit updated files:
wrglctl repos sync
Specify sync root folder:
wrglctl repos sync my_data_folder