How to use bazel import argument py_library
I am trying to use Boto3 in a Bazel project but cannot get the correct import for the library. Because of the Boto git repository, all sources are in named folders botocore
and boto3
at the root of the repository. Everything is imported boto3.boto3
, with the first corresponding to the name of the external dependency, and the second to the root folder in which it is located. How to use imports
rules attribute py_binary
and py_library
to import from internal boto3
instead of another?
This is what my workspace looks like:
//WORKSPACE
BOTOCORE_BUILD_FILE = """
py_library(
name = "botocore",
srcs = glob([ "botocore/**/*.py" ]),
imports = [ "botocore" ],
visibility = [ "//visibility:public" ],
)
"""
_BOTO3_BUILD_FILE = """
py_library(
name = "boto3",
srcs = glob([ "boto3/**/*.py" ]),
imports = [ "boto3" ],
deps = [ "@botocore//:botocore" ],
visibility = [ "//visibility:public" ],
)
"""
new_git_repository(
name = "botocore",
commit = "cc3da098d06392c332a60427ff434aa51ba31699",
remote = "https://github.com/boto/botocore.git",
build_file_content = _BOTOCORE_BUILD_FILE,
)
new_git_repository(
name = "boto3",
commit = "8227503d7b1322b45052a16b197ac41fedd634e9", # 1.4.4
remote = "https://github.com/boto/boto3.git",
build_file_content = _BOTO3_BUILD_FILE,
)
//BUILD
py_binary(
name = "example",
srcs = [ "example.py" ],
deps = [
"@boto3//:boto3",
],
)
//example.py
import boto3
boto3.client('')
Checking the contents of the assembly folder
$ ls bazel-bin/example.runfiles/*
bazel-bin/example.runfiles/__init__.py bazel-bin/example.runfiles/MANIFEST
bazel-bin/example.runfiles/boto3:
boto3 __init__.py
bazel-bin/example.runfiles/botocore:
botocore __init__.py
When I try to run the example script I get AttributeError: 'module' object has no attribute 'client'
I can import boto3.boto3
, but then using anything in it results in no dependencies like boto3.sessions
because everything is nested in<target-name>.boto3
source to share
I think you are on the right track, but you are facing a subtle problem due to the ordering of python sys.path.
If I run your example and print sys.path to example.py, I see that the path contains in the following order:
bazel-out/local-fastbuild/bin/example.runfiles
bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3
bazel-out/local-fastbuild/bin/example.runfiles/boto3
The second line is linked to imports = ['boto3']
in the WORKSPACE.
I think you want the third line where you get import boto3
it because you want python to see bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3/__init__.py
.
So, when python evaluates import boto3
, it sees bazel-out/local-fastbuild/bin/example.runfiles/boto3/__init__.py
from the first entry and uses that instead bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3/__init__.py
from the third entry.
I think the answer here is to call your "workspace" something other than the directory it contains. For example:
# WORKSPACE
new_git_repository(
name = "boto3_archive",
commit = "8227503d7b1322b45052a16b197ac41fedd634e9", # 1.4.4
remote = "https://github.com/boto/boto3.git",
build_file_content = _BOTO3_BUILD_FILE,
)
# BUILD
py_binary(
name = "example",
srcs = [ "example.py" ],
deps = [
"@boto3_archive//:boto3",
],
)
When I do this in your example, I get the following error: ImportError: No module named dateutil.parser
which I think is progress.
source to share