Working with external dependencies

Bazel can depend on targets from other projects. Dependencies from these other projects are called external dependencies.

The WORKSPACE file (or WORKSPACE.bazel file) in the workspace directory tells Bazel how to get other projects’ sources. These other projects can contain one or more BUILD files with their own targets. BUILD files within the main project can depend on these external targets by using their name from the WORKSPACE file.

For example, suppose there are two projects on a system:

/
  home/
    user/
      project1/
        WORKSPACE
        BUILD
        srcs/
          ...
      project2/
        WORKSPACE
        BUILD
        my-libs/

If project1 wanted to depend on a target, :foo, defined in /home/user/project2/BUILD, it could specify that a repository named project2 could be found at /home/user/project2. Then targets in /home/user/project1/BUILD could depend on @project2//:foo.

The WORKSPACE file allows users to depend on targets from other parts of the filesystem or downloaded from the internet. Users can also write custom repository rules to get more complex behavior.

This WORKSPACE file uses the same syntax as BUILD files, but allows a different set of rules. The full list of built-in rules are in the Build Encyclopedia’s Workspace Rules and the documentation for Embedded Starlark Repository Rules.

Like in the workspace directory, Bazel also supports WORKSPACE.bazel file as an alias of WORKSPACE in external dependencies. If both files exist, WORKSPACE.bazel will take the priority.

Supported types of external dependencies

A few basic types of external dependencies can be used:

Dependencies on other Bazel projects
Dependencies on non-Bazel projects
Dependencies on external packages

Depending on other Bazel projects

If you want to use targets from a second Bazel project, you can use local_repository, git_repository or http_archive to symlink it from the local filesystem, reference a git repository or download it (respectively).

For example, suppose you are working on a project, my-project/, and you want to depend on targets from your coworker’s project, coworkers-project/. Both projects use Bazel, so you can add your coworker’s project as an external dependency and then use any targets your coworker has defined from your own BUILD files. You would add the following to my_project/WORKSPACE:

local_repository(
    name = "coworkers_project",
    path = "/path/to/coworkers-project",
)

If your coworker has a target //foo:bar, your project can refer to it as @coworkers_project//foo:bar. External project names must be valid workspace names, so _ (valid) is used to replace - (invalid) in the name coworkers_project.

Depending on non-Bazel projects

Rules prefixed with new_, e.g., new_local_repository, allow you to create targets from projects that do not use Bazel.

For example, suppose you are working on a project, my-project/, and you want to depend on your coworker’s project, coworkers-project/. Your coworker’s project uses make to build, but you’d like to depend on one of the .so files it generates. To do so, add the following to my_project/WORKSPACE:

new_local_repository(
    name = "coworkers_project",
    path = "/path/to/coworkers-project",
    build_file = "coworker.BUILD",
)

build_file specifies a BUILD file to overlay on the existing project, for example:

cc_library(
    name = "some-lib",
    srcs = glob(["**"]),
    visibility = ["//visibility:public"],
)

You can then depend on @coworkers_project//:some-lib from your project’s BUILD files.

Depending on external packages

Maven artifacts and repositories

Use the ruleset rules_jvm_external to download artifacts from Maven repositories and make them available as Java dependencies.

Fetching dependencies

By default, external dependencies are fetched as needed during bazel build. If you would like to prefetch the dependencies needed for a specific set of targets, use bazel fetch. To unconditionally fetch all external dependencies, use bazel sync. As fetched repositories are stored in the output base, fetching happens per workspace.

Shadowing dependencies

Whenever possible, it is recommended to have a single version policy in your project. This is required for dependencies that you compile against and end up in your final binary. But for cases where this isn’t true, it is possible to shadow dependencies. Consider the following scenario:

myproject/WORKSPACE

workspace(name = "myproject")

local_repository(
    name = "A",
    path = "../A",
)
local_repository(
    name = "B",
    path = "../B",
)

A/WORKSPACE

workspace(name = "A")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
    name = "testrunner",
    urls = ["https://github.com/testrunner/v1.zip"],
    sha256 = "...",
)

B/WORKSPACE

workspace(name = "B")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
    name = "testrunner",
    urls = ["https://github.com/testrunner/v2.zip"],
    sha256 = "..."
)

Both dependencies A and B depend on testrunner, but they depend on different versions of testrunner. There is no reason for these test runners to not peacefully coexist within myproject, however they will clash with each other since they have the same name. To declare both dependencies, update myproject/WORKSPACE:

workspace(name = "myproject")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
    name = "testrunner-v1",
    urls = ["https://github.com/testrunner/v1.zip"],
    sha256 = "..."
)
http_archive(
    name = "testrunner-v2",
    urls = ["https://github.com/testrunner/v2.zip"],
    sha256 = "..."
)
local_repository(
    name = "A",
    path = "../A",
    repo_mapping = {"@testrunner" : "@testrunner-v1"}
)
local_repository(
    name = "B",
    path = "../B",
    repo_mapping = {"@testrunner" : "@testrunner-v2"}
)

This mechanism can also be used to join diamonds. For example if A and B had the same dependency but call it by different names, those dependencies can be joined in myproject/WORKSPACE.

Using Proxies

Bazel will pick up proxy addresses from the HTTPS_PROXY and HTTP_PROXY environment variables and use these to download HTTP/HTTPS files (if specified).

Transitive dependencies

Bazel only reads dependencies listed in your WORKSPACE file. If your project (A) depends on another project (B) which list a dependency on a third project (C) in its WORKSPACE file, you’ll have to add both B and C to your project’s WORKSPACE file. This requirement can balloon the WORKSPACE file size, but hopefully limits the chances of having one library include C at version 1.0 and another include C at 2.0.

Caching of external dependencies

By default, Bazel will only re-download external dependencies if their definition changes. Changes to files referenced in the definition (e.g., patches or BUILD files) are also taken into account by bazel.

To force a re-download, use bazel sync.

Layout

External dependencies are all downloaded to a directory under the subdirectory external in the output base. In case of a local repository, a symlink is created there instead of creating a new directory. You can see the external directory by running:

ls $(bazel info output_base)/external

Note that running bazel clean will not actually delete the external directory. To remove all external artifacts, use bazel clean --expunge.

Offline builds

It is sometimes desirable or necessary to run a build in an offline fashion. For simple use cases, e.g., traveling on an airplane, prefetching the needed repositories with bazel fetch or bazel sync can be enough; moreover, the using the option --nofetch, fetching of further repositories can be disabled during the build.

For true offline builds, where the providing of the needed files is to be done by an entity different from bazel, bazel supports the option --distdir. Whenever a repository rule asks bazel to fetch a file via ctx.download or ctx.download_and_extract and provides a hash sum of the file needed, bazel will first look into the directories specified by that option for a file matching the basename of the first URL provided, and use that local copy if the hash matches.

Bazel itself uses this technique to bootstrap offline from the distribution artifact. It does so by collecting all the needed external dependencies in an internal distdir_tar.

However, bazel allows the exeuction of arbitrary commands in repository rules, without knowing if they call out to the network. Therefore, bazel has no option to enforce builds being fully offline. So testing if a build works correctly offline requires external blocking of the network, as bazel does in its bootstrap test.

Best practices

Repository rules

Prefer http_archive to git_repository and new_git_repository.

Do not use bind(). See “Consider removing bind” for a long discussion of its issues and alternatives.

Repository rules

A repository rule should generally be responsible for:

Detecting system settings and writing them to files.
Finding resources elsewhere on the system.
Downloading resources from URLs.
Generating or symlinking BUILD files into the external repository directory.

Avoid using repository_ctx.execute when possible. For example, when using a non-Bazel C++ library that has a build using Make, it is preferable to use repository_ctx.download() and then write a BUILD file that builds it, instead of running ctx.execute(["make"]).