Installing pybedtools

This may actually be useful for any python package depending in pysam (or pysam itself).

I wanted to use the pybedtools python package on the cluster. I tried to install it using pip3.6 (after module load Python/3.6.0) but the installation failed as follows

Code Block (text)

configure: error: liblzma development files not found

The CRAM format may use LZMA2 compression, which is implemented in HTSlib
by using compression routines from liblzma <http://tukaani.org/xz/>.

Building HTSlib requires liblzma development files to be installed on the
build machine; you may need to ensure a package such as liblzma-dev (on Debian
or Ubuntu Linux), xz-devel (on RPM-based Linux distributions or Cygwin), or
xz (via Homebrew on macOS) is installed; or build XZ Utils from source.

Either configure with --disable-lzma (which will make some CRAM files
produced elsewhere unreadable) or resolve this error to build HTSlib.
make: ./version.sh: Command not found
make: ./version.sh: Command not found
config.mk:2: *** Resolve configure error first.  Stop.
# pysam: no cython available - using pre-compiled C
# pysam: htslib mode is shared
# pysam: HTSLIB_CONFIGURE_OPTIONS=None
# pysam: htslib configure options: None
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/local/scratch/pip-install-7yi14k3t/pysam/setup.py", line 223, in <module>
    htslib_make_options = run_make_print_config()
  File "/local/scratch/pip-install-7yi14k3t/pysam/setup.py", line 69, in run_make_print_config
    stdout = subprocess.check_output(["make", "-s", "print-config"])
  File "/local/gensoft2/exe/Python/3.6.0/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/local/gensoft2/exe/Python/3.6.0/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['make', '-s', 'print-config']' returned non-zero exit status 2.

This seems to be a problem with pysam, actually.

Step-by-step guide#

The following worked for me:

Load the python and xz modules

Code Block (bash)

module load Python/3.6.0 xz

Install pybedtools using pip

Code Block (bash)

python3.6 -m pip install --user pybedtools

It may happen that pip3.6 fails as follows

Code Block (bash)

Traceback (most recent call last):

  File "/local/gensoft2/exe/Python/3.6.0/bin/pip3.6", line 7, in <module>

    from pip import main

ImportError: cannot import name 'main'

This can be solved using python3.6 -m pip instead of pip3.6 (which is what I did in the step 2 above).

Solving import errors#

When importing pybedtools (or, actually, pysam) in python, the following error may occur

Code Block (bash)

ImportError: liblzma.so.5: cannot open shared object file: No such file or directory

This can be solved by issuing the following command before starting python:

Code Block (bash)

export LD_LIBRARY_PATH="/local/gensoft2/exe/xz/5.2.2/lib"