ConceptsIntegrationJavaScript SDKREST APIWrglctl reference

Wrglctl reference

Wrglctl is our official command-line client. It is meant for CI use cases but you can also use it as a convenient way to interact with WRGL from the command line. It allows you to create commit, download CSV, show commit logs and more.

Download

Download & install instruction for each OS.

Environment variables

When using Wrglctl in CI it's easier to login using these environment variables. Note that you need to set both WRGL_APIKEY and WRGL_USERNAME for operations to work smoothly.

Env var Description
WRGL_APIKEY The API key to authenticate your requests.
WRGL_USERNAME Your username.

Login

You can login with email/password or an API key. In either case an API key will be saved in the machine for subsequent runs.

Usage

wrglctl login [flags]

Flags

Full Short Description
--apikey -k API key string.

Create commit

Commit a CSV file and possibly create the repo if it does not already exist.

Usage

wrglctl repos commit CSV_FILE_PATH REPO_NAME MESSAGE [flags]

Arguments

Arguments Description
CSV_FILE_PATH Path to the CSV file.
REPO_NAME Repository name. If the repo already exist then append this commit, otherwise create it.
MESSAGE Commit's purpose in plain English.

Flags

Full Short Description
--primary-key -p Specify which columns are primary key. Multiple columns can be separated by comma. e.g. "pid,year". If primary key is not specified then all columns are included in the primary key.
--validate-row-uniqueness -i Print an error if there are rows with duplicated primary key.

List repositories

Show all repositories.

Usage

wrglctl repos list [flags]

Flags

Full Short Description
--no-pager -P Don't use pager (specified by env var PAGER) to paginate output. Print everything at once.

Show commit logs

Show commit logs for a repo. Output very similar to git log.

Usage

wrglctl repos log REPO_NAME [flags]

Arguments

Arguments Description
REPO_NAME Name of repository to show logs.

Flags

Full Short Description
--no-pager -P Don't use pager (specified by env var PAGER) to paginate output. Print everything at once.

Pull CSV

Download the latest commit from a repo and save to a local CSV file. You can also pull an earlier commit by specifying a hash (can be looked up with wrglctl repos log).

Usage

 wrglctl repos pull REPO_NAME OUTPUT_FILE [flags]

Arguments

Arguments Description
REPO_NAME Name of repository to pull from.
OUTPUT_FILE Save CSV to this file.

Flags

Full Short Description
--commithash -H Specify the exact commit to pull.

Sync multiple repos

When you work with multiple repositories it is easy to lose track of which file commit to which repo, which repos are up-to-date, etc. You can write a script to automate some of that but it still isn't easy to get right. repos sync command will help with that by keeping track of multiple repos and only pull newer files. And when you make changes to those files locally, it look at file's mod time and will only commit the files that you updated. Furthermore a sync file can be linked to a parent Git repo from which it will read Git commit info and put that in the WRGL commit message.

This command generate 2 files in sync root folder:

  • .wrglsync.yaml: This file is called "sync file". It configures behavior of sync command. You will want to commit this file in your Git repo.
  • .wrglmod.yaml: This file is called "mod file" and it keep track of modification time for each file so that only updated files are committed during sync. You will want to add this file to .gitignore as to not commit it to Git repo.

Usage

wrglctl repos sync [DIRECTORY] [flags]

Arguments

Arguments Description
DIRECTORY Optional. This is the sync root directory. If not specified then it is current working directory.

Sync file

When you run repos sync for the first time. You will be asked a series of questions to create a sync file with the name .wrglsync.yaml in a folder of your choosing which will also be the root folder for files that need to be synced.

Fields Description
username The username that the repos belong to. This field become relevant when team feature is introduced.
repos A map of repos managed by sync command. Each key is the repo name while the value is a repo sync object.
commit Optional, only add when you wish for repos sync to commit files for you as well.
commit.messageTemplate This is a template for commit message written in Go template language. For now its only use case is to add info from Git commit to your WRGL commit message. Therefore only the following keys are available:
  • GitCommit.Message: Git commit message.
  • GitCommit.Link: If field commit.gitRemote is not empty then this field will be the full link to the Git commit on that host. Provided you have pushed the changes of course.
  • GitCommit.AuthorName: Name of Git commit author.
  • GitCommit.AuthorEmail: Email of Git commit author.
commit.gitRoot The Git repo directory from which Git info will be read (and linked to WRGL commit), must be one of the parent folders to the sync root folder. If you create sync file with repos sync then it will help you select the correct folder.
commit.gitRemote This field tell repos sync command how to construct GitCommit.Link during commit. The recommended way to populate this field is to let repos sync command fill it as part of sync file creation. It should work with major Git hosts such as GitHub, GitLab, BitBucket, Azure, ... Though for now only GitHub is well-tested.

Repo sync object

Fields Description
fileName Relative path to the file that will be committed/updated for this repo.
primaryKey Optional, this is the list of columns included in primary key when repo is committed.

Create sync file in current folder if none exist:

wrglctl repos sync

Example content of sync file:

username: ipno
repos:
  personel:
    fileName: fuse/personel.csv
    primaryKey:
      - uid
  personel_history:
    fileName: fuse/personel_history.csv
  complaint:
    fileName: fuse/complaint.csv
commit:
  messageTemplate: |-
    {{.GitCommit.Link}}

    {{.GitCommit.Message}}
  gitRemote:
    host: github.com
    dir: ppact/processing
  gitRoot: ..

Pull newer files and commit updated files:

wrglctl repos sync

Specify sync root folder:

wrglctl repos sync my_data_folder