A private PyPI server lets you host your own Python packages internally, keeping sensitive code off public repositories and speeding up dependency resolution for your team.
Here’s how to set up and use one, focusing on devpi, a popular choice.
First, you need to install devpi-server and devpi-client on a machine that will host your index.
pip install devpi-server devpi-client
Now, initialize the server. This creates the necessary configuration and data directories.
devpi-server --init
Start the server. It typically runs on port 3141.
devpi-server --host 0.0.0.0 --port 3141
You’ll see output indicating the server is running. You can access the web interface at http://your-server-ip:3141.
Next, configure your local pip to use this private index. This involves setting up authentication and telling pip where to find your index.
First, log in to your devpi server using the client. You’ll need to create a user.
devpi-client --host your-server-ip --port 3141 login --username root --password <your-root-password>
(If you haven’t set a root password during devpi-server --init, devpi will prompt you to create one. For security, it’s best to set a strong password.)
Once logged in as root, create a new user for your team.
devpi-client --host your-server-ip --port 3141 user -c myuser password=mypwd
Now, create a "stage" for your packages. A stage is like a namespace for your index.
devpi-client --host your-server-ip --port 3141 index -c my-index bases=root/pypi myuser/my-index
This creates an index named my-index under your user myuser, and it inherits packages from the public root/pypi index. This means you can still access public packages through your private server.
To make pip use this index, you need to create a pip.conf (or pip.ini on Windows) file. The location depends on your OS:
- Linux/macOS:
$HOME/.config/pip/pip.confor$HOME/.pip/pip.conf - Windows:
%APPDATA%\pip\pip.ini
Add the following to your pip.conf/pip.ini:
[global]
index-url = http://myuser:mypwd@your-server-ip:3141/my-index/simple/
extra-index-url = http://pypi.org/simple/
The index-url points directly to your private index. The extra-index-url is crucial for allowing pip to fall back to the public PyPI if a package isn’t found on your private server.
Now, if you try to install a package that you’ve uploaded to your private index, pip will fetch it from there.
To upload a package, you first need to build it. Navigate to your package’s source directory and run:
python setup.py sdist bdist_wheel
Then, use devpi-push to upload it to your private index:
devpi-push --host your-server-ip --port 3141 --username myuser --password mypwd my-index dist/*
This command uploads all files in the dist/ directory to the my-index stage.
If you want to mirror public packages, devpi does this automatically by default when you create a new index with bases=root/pypi. When a package is requested from your private index that isn’t there, devpi fetches it from root/pypi and caches it locally. Subsequent requests will be served from your cache, significantly speeding up builds.
The extra-index-url in your pip.conf is what allows pip to seamlessly switch between your private index and the public PyPI. pip checks the index-url first. If it doesn’t find the requested package, it then checks the extra-index-urls in order. This ensures you can install both internal and external dependencies with a single pip install command.
When devpi mirrors packages from root/pypi, it doesn’t just store the package file; it also stores metadata. This means when pip queries your private index for a package, devpi can respond with the available versions and metadata directly from its cache, without needing to hit the public PyPI. This is why dependency resolution becomes much faster.
If you later decide to remove the extra-index-url from your pip.conf and only use your private index, pip install will fail for any package not present on your private server. This is a common way to enforce that all dependencies must be approved and hosted internally.
The next thing you’ll likely want to explore is setting up CI/CD pipelines to automatically build and upload your internal packages to devpi.