Python/pip

pip

pip - installs packages. Python packages. An easy_install replacement

pip 1.1 : Python Package Index - http://pypi.python.org/pypi/pip

Usage:

pip help
pip install [package]   # Example: 'pip install test'
pip -v --upgrade install [package]   # force upgrade
pip uninstall [package]
pip search [package]
pip freeze              # list all currently installed pip packages
pip list                # also list all packages
pip show [installed_package]      # minimal info about installed package

PIP Source Code: http://pypi.python.org/packages/source/p/pip/

Package Destination: /usr/lib/python2.4/site-packages/

Test package example:

• Package Page: http://pypi.python.org/pypi/test
• Package Source Code: http://pypi.python.org/packages/source/t/test/test-2.3.4.5.tar.gz
• Package Destination: /usr/lib/python2.4/site-packages/test-2.3.4.5-py2.4.egg-info/

Installation

Linux and Windows:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py

ref: https://github.com/pypa/get-pip

curl -sSL https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py

--

Installing the Package Tools — The Hitchhiker's Guide to Packaging v1.0 documentation - http://guide.python-distribute.org/installation.html

• Pip Installs Python (Pip)

Pip is an installer for Python packages written by Ian Bicking. It can install packages, list installed packages, upgrade packages and uninstall packages. The pip application is a replacement for easy_install. It uses mostly the same techniques for finding packages, so packages that were made easy_installable should be pip-installable as well.

Dependency Installation:

# dependency setuptools
# Python Package Index : setuptools 0.6c11 - http://pypi.python.org/pypi/setuptools
# source code: http://pypi.python.org/packages/source/s/setuptools/
# VER=1.1.6
VER=2.0
mkdir -p ~/.src ; cd ~/.src
wget --no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools-$VER.tar.gz
tar -zvxf setuptools-$VER.tar.gz
cd setuptools-$VER
sudo python setup.py install

# yum method, if available
yum install python-setuptools

Easy Install:

sudo easy_install pip

Windows:

C:\python27\scripts\easy_install.exe pip

Apt:

sudo apt-get install python-setuptools
sudo easy_install pip

Manual Installation:

PIP_VER=1.4.1
mkdir -p ~/.src ; cd ~/.src
# Python Package Index : pip 1.0.1 - http://pypi.python.org/pypi/pip#downloads
wget http://pypi.python.org/packages/source/p/pip/pip-$PIP_VER.tar.gz
tar -zvxf pip-$PIP_VER.tar.gz
cd pip-$PIP_VER 
sudo python setup.py install

# fix pip:
sudo sed -i 's%#!/usr/bin/python%#!/usr/bin/env python%' /opt/python26/bin/pip

NOTE: Most packages failed during installation. The simple 'test' package installed though.

Packages are generally installed into:

/usr/local/lib/python2.7/dist-packages/

PyPI - Create PIP Package

Reference

• Packaging Python Projects - Python Packaging User Guide - https://packaging.python.org/en/latest/tutorials/packaging-projects/

Create the following file structure locally:

packaging_tutorial/
└── src/
    └── example_package_YOUR_USERNAME_HERE/
        ├── __init__.py
        └── example.py

---

my_package/
├── LICENSE.txt
├── README.md
├── my_package/
│   ├── __init__.py
│   ├── module1.py
│   └── module2.py
└── tests/
    └── test_module1.py

Create a pyproject.toml file at the root of your project. This file specifies build system requirements and project metadata.

[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "my_package"
version = "0.1.0"
authors = [
  { name="Your Name", email="your.email@example.com" },
]
description = "A short description of your package."
readme = "README.md"
requires-python = ">=3.7"
classifiers = [
    "Programming Language :: Python :: 3",
    "License :: OSI Approved :: MIT License",
    "Operating System :: OS Independent",
]
dependencies = [
    "some_dependency>=1.0",
    "another_dependency",
]

Build:

• Install the build package: pip install build
• Navigate to your project's root directory in the terminal.
• Run the build command: python -m build
• This will create dist/ directory containing your source distribution (.tar.gz) and a wheel distribution (.whl).

---

setup.py: [1]

from distutils.core import setup

setup(
    name='kentest',
    version='0.1.0',
    author='Kenneth Burgener',
    author_email='kenneth@oeey.com',
    packages=['kentest'],
    url='http://pypi.python.org/pypi/kentest/',
    license='LICENSE.txt',
    description='kentest.',
    long_description=open('README.txt').read(),
)

Create distribution package:

# output to: dist/[PROJECT]-[VERSION].tar.gz
python setup.py sdist

Test install:

sudo pip install dist/[PROJECT]-[VERSION].tar.gz

Remove Test install:

sudo pip uninstall [PROJECT]

-- New Upload Process --

Install Twine:

python3 -m pip install --upgrade twine

Upload to test repository:

python3 -m twine upload --repository testpypi dist/*

Upload to main repository:

python3 -m twine upload dist/*

ref: https://packaging.python.org/tutorials/packaging-projects/

-- Old Upload Process --

Configure PyPI settings:

# ~/.pypirc
python setup.py register

Upload project:

python setup.py sdist upload

-- Install --

Install project:

pip install kentest

---

Sections:

Scripts:

scripts=['bin/mytool']

Data files: ("*" doesn't appear to work)

     data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
                 ('config', ['cfg/data.cfg']),
                 ('/etc/init.d', ['init-script'])]

Package files:

     packages=['mypkg'],
     package_dir={'mypkg': 'src/mypkg'},
     package_data={'mypkg': ['data/*.dat']},

References:

• The Python Package Index (PyPI) — Python documentation - https://docs.python.org/2/distutils/packageindex.html
• Writing the Setup Script — Python documentation - https://docs.python.org/2/distutils/setupscript.html
• Creating a Package — The Hitchhiker's Guide to Packaging 1.0 documentation - http://guide.python-distribute.org/creation.html

---

reStructuredText

PyPI uses reStructuredText instead of Markdown (shame!)

Install pypandoc:

yum install pandoc
pip install pypandoc

To have setup.py auto convert it, install pandoc and add the following: [2]

from setuptools import setup
try:
    from pypandoc import convert
    read_md = lambda f: convert(f, 'rst')
except ImportError:
    print("warning: pypandoc module not found, could not convert Markdown to RST")
    read_md = lambda f: open(f, 'r').read()

setup(
    # name, version, ...
    long_description=read_md('README.md'),
    install_requires=[]
)

This will automatically convert README.md to RST for the long description using on PyPi. When pypandoc is not available, then it just reads README.md without the conversion – to not force others to install pypandoc when they wanna just build the module, not upload to PyPi.

So you can write in Markdown as usual and don’t care about RST mess anymore. :-)

rst test tool: http://rst.ninjs.org/

References:

• http://stackoverflow.com/questions/10718767/have-the-same-readme-both-in-markdown-and-restructuredtext

PyPI - Create Pip Repo

Create a directory structure.

    my_pypi_repo/
    ├── package_a/
    │   ├── package_a-1.0.0-py3-none-any.whl
    │   └── package_a-1.0.0.tar.gz
    └── package_b/
        ├── package_b-2.1.0-py3-none-any.whl
        └── package_b-2.1.0.tar.gz
    └── package_c/
        ├── package_c-1.0.0-py3-none-any.whl
        ├── package_c-1.0.0.zip
        ├── package_c-2.1.0-py3-none-any.whl
        └── package_c-2.1.0.zip

Each package should have its own subdirectory within the main repository directory.

Serve the directory: Use a web server (e.g., Apache, Nginx, or Python's built-in http.server) to serve the my_pypi_repo directory with auto indexing enabled.

Configure pip on clients.

    # ~/.config/pip/pip.conf or /etc/pip.conf
    [global]
    extra-index-url = http://your_server_ip:8000/

Or specify url directly on installation:

pip install package_a --extra-index-url http://your_server_ip:8000/

Reference:

• Hosting your own simple repository - Python Packaging User Guide - https://packaging.python.org/en/latest/guides/hosting-your-own-index/

Multiple PIP Archive URLs

Multiple PIP Archive URLs

[global]
index-url = https://somedomain.org/simple
trusted-host = somedomain.org
               pypi.org
               secondary.extra.host
extra-index-url= http://pypi.org/simple <= either one of these is fine
                 https://pypi.org/simple <= either one of these is fine
                 http://secondary.extra.host/simple

Pip Repo Password

Pip Authentication
https://pip.pypa.io/en/stable/topics/authentication/

Basic HTTP authentication

pip supports basic HTTP-based authentication credentials. This is done by providing the username (and optionally password) in the URL:

https://username:password@pypi.company.com/simple
For indexes that only require single-part authentication tokens, provide the token as the “username” and do not provide a password:

https://0123456789abcdef@pypi.company.com/simple

netrc support

Below is an example .netrc, for the host example.com, with a user named daniel, using the password qwerty:

machine example.com
login daniel
password qwerty

Keyring Support

Pip Keyring Support
https://pip.pypa.io/en/stable/topics/authentication/#keyring-support

pip supports loading credentials stored in your keyring using the keyring library, which can be enabled py passing --keyring-provider with a value of auto, disabled, import, or subprocess. The default value auto respects --no-input and does not query keyring at all if the option is used; otherwise it tries the import, subprocess, and disabled providers (in this order) and uses the first one that works.

Issues

Ubuntu 24 pip packages

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.