Setting Up Pants
Pants will work out of the box, at least for basic functionality. However some uses require further setup.
Source Roots
Pants needs to know where the root of a source tree is, so it can correctly set up import paths, bundle code etc.
In typical cases, Pants can deduce the source roots automatically based on naming conventions.
E.g., src/<lang>, src/main/<lang>, test/<lang>, src/test/<lang>, 3rdparty/<lang> and so on.
However if your source roots don't conform to any of the default patterns, you can add your own
patterns. See ./pants help-advanced source for details.
.gitignore
If you use Git as your SCM you'll want to add the following entries to your .gitignore file:
.pants.d
dist
.pids
.pants.workdir.file_lock
BUILD files
BUILD files describe the dependency graph of your codebase. You'll need to create some in order to use Pants.
If you're converting a mvn-built workspace to Pants, you can get a head start by using
information from your pom.xml files. The mvn2pants
repo demonstrates some scripts that convert pom.xml files into BUILD files. This can get
you up and running with Pants faster. Careful, though: Maven projects tend to be very coarse-grained.
Once you have things working, you'll probably want to break down the large Maven-style targets
into finer-grained ones.
Options
Pants is very configurable and has literally hundreds of tuneable parameters, known in Pants parlance as options.
Most options are automatically set to useful defaults. However it is inevitable that sooner or later you'll need to tweak some of them.
Tools
Pants calls out to other tools for many build steps. E.g., it uses Zinc for incremental compilation of Java and Scala code.
Pants bootstraps such tools as needed. The default versions of these tools can be overridden
using the appropriate options. It's idiomatic, but not required, to set these options to addresses
of targets in a special BUILD file named BUILD.tools in the repo root.
Pants uses Ivy to fetch JVM-based tools. Ivy requires an XML configuration file.
By default, pants uses the configuration that ships with Ivy,
but if you need to change Ivy settings you can specify your own in pants.toml:
[ivy] ivy_settings = "%(pants_supportdir)s/ivy/ivysettings.xml" cache_dir = "~/.ivy2/pants"
For more information on Ivy settings, see the Ivy documentation
Build Cache
Pants automatically caches its work locally. But most organizations will want to set up a shared
remote build cache. To do so, set cache options in pants.toml:
[cache] read_from = ['https://myserver.co/pantscache'] write_to = ['https://myserver.co/pantscache']
Note that the read and write cache locations are separate options: you may, for example, only want your CI machines, not individual developer builds, to populate the shared cache.
Valid option values are:
[ 'https://myserver.co/pantscache' ]RESTful server URL.[ '/tmp/pantscache' ]local directory.[ 'https://myserver.us/pcache|https://myserver.bf/pcache' ]pipe-separated list of URLs--Pants pings each and uses the fastest.[ '/tmp/pantscache', 'https://myserver.co/pcache' ]try local fs first, then remote server.
For details see the _do_create_artifact_cache docstring in
cache_setup.py.
The remote server should be a RESTful server that can handle GET, HEAD, PUT, and DELETE requests
on large binary blobs. E.g., you can implement this using Varnish
in front of nginx (with dav_methods PUT DELETE; enabled).
When reading from a cache server, Pants tries to GET an URL at some path under the server URL;
your server should respond with 404 if there's nothing there, 200 if there is.
When writing to a cache server, Pants first tries a HEAD request to see if the file's already
there; respond with 404 if there's nothing there, 200 if there is. If Pants gets a 404, it will
then try a PUT request; store the file it sends.
If the user's .netrc has authentication information for the cache server[s], Pants will use it.
(Thus, if only some users with known-good setups should be able to write to the cache, you might
find it handy to use .netrc to authenticate those users.)
Uploading Timing Stats
Pants tracks information about its performance: what it builds, how much time various build operations take, cache hits, and more. If you you work with a large engineering organization, you might want to gather this information in one place, so it can inform decisions about how to improve everybody's build times.
To upload these stats to a server for future analysis set the following option in pants.toml:
[run-tracker] stats_upload_urls = "{'http://myorg.org/pantsstats': 'authprovider'}"
Pants will POST JSON data to each key URL. The authprovider is the name of the provider the
user must auth against in order to post to the corresponding URL.
See here for details on how to authenticate Pants against
an auth provider.
If posting to the URL does not require authentication, use the empty string as the auth provider.
The posted JSON format should be self-explanatory.
Using Pants behind a firewall
Pants may encounter issues running behind a firewall. Several components expect to be able to reach the Internet:
- Ivy bootstrapper
- Binary tool bootstrapping
- Ivy itself (used for tool bootstrapping and downloading external .jar files)
- Python requirements
Configuring the Python requests library
Code in bootstrapper.py and other parts of Pants use the Python
requests library to
download resources using http or https. The first time you may
encounter this is when Pants attempts to download an initial version
of Ivy. If this initial download is through a proxy, the requests
library uses the HTTP_PROXY or HTTPS_PROXY environment variable to
find the proxy server.
export HTTP_PROXY=http://proxy.corp.example.com:123 export HTTPS_PROXY=https://proxy.corp.example.com:456
If Pants tries to find resources with HTTPS urls, you may see an error like:
Exception message: Problem fetching the ivy bootstrap jar! Problem GETing data from https://artifactserver.example.com/content/groups/public/org/apache/ivy/ivy/2.3.0/ivy-2.3.0.jar: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
The requests library attempts to verify SSL certificates by default.
The reason it is denying the request is that it cannot find a trusted
public key for the root certificate authority presented by the server.
The requests library uses the 'certifi' library of well known
certificate authorities, if that library is installed. If you are
using a virtualenv installation of pants, using pip
install certifi to add the certify package to your pants environment
might help. You can also download a .pem bundle from the
certifi project page and set
REQUESTS_CA_BUNDLE as mentioned below.
If you are using hosts with a self-signed certificate, your
certificate authority will not be available in the certifi library.
You will need a .pem file for the local certificate authority.
You can tell the requests library about your trusted certificate
authority certificates by setting the environment variable REQUESTS_CA_BUNDLE.
This variable should point to a file containing trusted certificates:
export REQUESTS_CA_BUNDLE=/etc/certs/latest.pem
Nexus as proxy
If your site uses Sonatype Nexus or another reverse proxy for artifacts, you do not need to use a
separate HTTP proxy. Contact the reverse proxy administrator to setup a proxy for the sites
listed in pants.toml. These are likely to be https://repo1.maven.org/maven2/ and
binaries.pantsbuild.org.
Redirecting tool downloads to other servers
For the binary support tools like protoc, you will need to setup a proxy for the
binaries.pantsbuild.org repo, or create your own repo of build tools:
binaries_baseurls = [ "https://nexus.example.com/content/repositories/binaries.pantsbuild.org", ]
Redirecting python requirements to other servers
For Python repos, you need to override the following settings in pants.toml:
[python-repos] repos = [ "https://wheels.example.com/content/repositories/", ] indexes = [ "https://pypi.org/simple/", ]
You can also reference a local repo relative to your project's build root with this pattern:
[python-repos] repos = [ "%(buildroot)s/repo_path", ]
Bootstrapping Ivy
For JVM publishing, pants fetches the Ivy tool with an initial manual bootstrapping step using the
Python requests library.
If you do not want to use HTTP_PROXY or HTTPS_PROXY as described above, you can re-redirect
this initial download to another URL with a setting in pants.toml:
[ivy] bootstrap_jar_url = "https://proxy.corp.example.com/content/groups/public/org/apache/ivy/ivy/2.3.0/ivy-2.3.0.jar"
You may also encounter issues downloading this .jar file if you are using self-signed SSL certificates. See the section on Configuring the Python requests library above.