Release Process For C++ Components

NOTES:

• This process covers major and minor releases only. Bug-fix releases, which increment the third digit, are performed on a A.B.X branch and not on master, but otherwise is similar.
• Some of these operations can be performed either on the Command-Line or in your IDE, whatever you prefer.

Preparation

• Confirm correctness for
  • LICENSE
  • NOTICE – check for copyright dates
  • README.md
  • .asf.yaml
  • .gitignore
• From Command Line or IDE:
  • Run Unit tests
  • Run Code Coverage > 90%
  • Confirm that all temporary branches are checked into master and/or deleted, both local and remote.
  • Confirm any new bug fixes have corresponding tests
• From Command Line at Component root:
  • To confirm gpg-agent is running type:
    • ps -axww | grep gpg # you should see something like:
      • 64438 ?? 0:30.33 gpg-agent –homedir /Users/<name>/.gnupg –use-standard-socket –daemon
    • To start GPG if GPG Agent is not running:
      • eval $(gpg-agent --daemon)
  • Confirm GitHub repository is current and git status is clean:
    • git status # should return:
    • “nothing to commit, working tree clean”
  • At major version releases, search for deprecated code and remove at Major Versions only.
    • find . -name "*.?pp" -type f -print | xargs grep -i -n -s -A0 "deprecated"
    • you may need to ignore false positives in the pybind11 directory

Create Permanent Release Branch & Python Version Preparation

• Assume target version = A.B.0
• From IDE or Command Line:
  • Create new Permanent Branch: “A.B.X”
  • Change the content of the version.cfg.in file to A.B.0
  • Commit the change.
  • Create Annotated TAG: A.B.0-RC1 (or RCn)
  • Write down the Git hash : example: 40c6f4f
  • Push Branch “A.B.X” with edited version.cfg.in to origin
  • DO NOT MERGE THIS PERMANENT BRANCH INTO MASTER
  • Do explicit push of tags on branch “A.B.X” to origin:
    • git push origin --tags
• From a web browser at origin web site: github.com/apache/datasketches-<component>
  • Select the A.B.X branch
  • Confirm that the tag: A.B.0-RC1 exists and that the tag is on the latest commit and with the correct Git hash.
  • DO NOT CREATE PR OR MERGE THIS PERMANENT BRANCH INTO MASTER
• From IDE or Command Line:
  • Confirm that the tag A.B.0-RC1 and the branch A.B.X, and HEAD coincide with the correct Git hash.
  • Confirm that there are no unstaged or staged changes.
  • Return to master branch
  • Edit master version.cfg.in to A’.B’.@DT@.@HHMM@ where A’ or B’ will be incremented by 1.
  • Commit and Push this change to origin/master with the comment “Release Process: Change version to A’.B’ development”
  • Return to release branch A.B.X

Create and/or Checkout Local dist/dev directories on your system

• If you have not already, on your system create the two directory structures that mirror the dist.apache.org/repos/ directories:
  • mkdir dist/dev/datasketches/
  • mkdir dist/release/datasketches/
• Checkout both “dev” and “release” directories
  • Open a terminal in the dist/dev/datasketches directory and do a checkout:
    • svn co https://dist.apache.org/repos/dist/dev/datasketches/ . #Note the DOT
    • svn status # make sure it is clean
  • Open a terminal in the dist/release/datasketches directory and do a checkout:
    • svn co https://dist.apache.org/repos/dist/release/datasketches/ . #Note the DOT
    • svn status # make sure it is clean

Create the Candidate Apache Release Distribution on dist/dev

Create primary zip files & signatures

• You will need the following arguments:
  • Absolute path of target project.basedir on your system
  • Artifact name : datasketches-<component> where component is e.g., cpp
  • GitHub Tag: A.B.0-RC1 (or RCn)
• Start a new terminal in the above dist/dev/datasketches/scripts directory on your system:
  • To confirm gpg-agent is running type:
    • ps -axww | grep gpg # you should see something like:
      • 64438 ?? 0:30.33 gpg-agent –homedir /Users/<name>/.gnupg –use-standard-socket –daemon
    • To start GPG if GPG Agent is not running:
      • eval $(gpg-agent --daemon)
  • Run something like:
    • ./bashDeployToDist.sh /Users/\<name\>/dev/git/Apache/datasketches-\<component\> datasketches-\<component\> A.B.0-RC1
    • Follow the instructions.
    • NOTE: if you get the error “gpg: signing failed: No pinentry”:
      • open .gnupg/gpg-agent.conf
      • change to: pinentry-program /usr/local/bin/pinentry-tty
      • reload the gpg agent in the terminal: gpg-connect-agent reloadagent /bye
      • restart the ./bashDeployToDist script
    • Close the terminal
• Check and grab the web URL ~ https://dist.apache.org/repos/dist/dev/datasketches/<component>/A.B.0-RC1/
  • There should be 3 files: *-src.zip, *-src.zip.asc, *-src.zip.sha512

Create Copy of External Artifact Distributions

• Run ‘Build Python Wheels’ action on GitHub for the release branch
• When the build finishes download artifact.zip from it
• Use sign_pypi_wheels.sh script to add GPG signatures and SHA512 checksums
• Check in the result as ‘pypi’ subdirectory in the release candidate directory

Prepare & Send [VOTE] Letter to dev@

• See VoteTemplates directory for a recent example
• If vote is not successful, fix the problem and repeat above steps.
• After a successful vote return to this point and continue …

Prepare & Send [VOTE-RESULT] Letter to dev@

• See VoteTemplates directory for a recent example
• Declare that the vote is closed.
• Summarize PPMC vote results

Move files from dev/staging to release

• use dist/dev/datasketches/scripts/moveDevToRelease.sh script to move the approved release candidate to the destination
• upload pypi artifacts to pypi.org
  • make a temporary copy of the pypi directory and remove *.asc and *.sha512 files from it
  • follow this guide
    • upload to test.pypi.org
    • install from test pypi
    • upload to pypi.org

Create & Document Release Tag on GitHub

• Open your IDE and switch to the recently created Release Branch A.B.X
• Find the recently created A.B.0-RCn tag in that branch
• At that same GitHub ID hash, create a new tag A.B.0 (without the RCn).
• From the Command Line: Push the new tag to origin:
  • git push origin --tags
• On the GitHub component site document the release

Update Website Downloads.md “Latest Source Zip Files” Table

• This script assumes that the remote …/dist/release/datasketches/… directories are up-to-date with no old releases.
• Start a new terminal in the ../dist/dev/datasketches/scripts directory on your system:
• Make sure you local website directory is pointing to master and up-to-date.
• Run the following with the argument specifying the location of your local website directory:
  • ./createDownloadsInclude.sh /Users/\<name\>/ ... /datasketches-website
• When this is done, be sure to commit the changes to the website.

Update Website Documentation

• Run Doxygen GitHub action for the release tag
• Check that it finished successfully and, optionally, check that it created docs/X.Y.Z directory in the “gh-pages” branch
• Check https://apache.github.io/datasketches-cpp/X.Y.Z/
• Update link in _includes/javadocs.html in the datasketches-website repository

Prepare Announce Letter to dev@

• ASF requests that you wait 24 hours to publish Announce letter to allow the propagation to mirrors.
• Use recent template
• Summarize vote results

Update These Instructions

• If you have updated this file or any of the scripts, please update this file on the website and dist/dev/datasketches for the scripts.