I’ve spent the day building a conda recipe, the process wasn’t super-smooth, hopefully these notes will help others and/or maybe you can leave me a comment to improve my flow. The goal was to learn how to use conda to distribute a package that ordinarily I’d put on PyPI .
I’m using Linux 64bit (Mint 18 on an XPS 9550), conda 4.1 and conda build 1.21.14 (up to date as of today). My goal was to build a recipe ( python_template_with_config_recipe ) to install my python_template_with_config (a bit of boilerplate code I use sometimes when making a new project). That template has two modules, a test, a setup.py and it depends on numpy.
The short story:
- git clone https://github.com/ianozsvald/python_template_with_config_recipe.git
- cd inside, run “conda build –debug .”
- # note the period means “current directory” and that’s two dashes debug
- a local bzip will be built and you’ll see that the 1 test ran ok
On my machine the built code ends up in “~/anaconda3/pkgs/python_template_with_config-0.1-py35_0/lib/python3.5/site-packages/python_template_with_config” and the building takes place in “~/anaconda3/conda-bld/linux-64”.
In a new conda environment I can use “conda install –use-local python_template_with_config” and it’ll install the built recipe into the new environment.
To get started with this I first made a fresh empty conda environment (note that anaconda isn’t my default Python, hence the long-form access to `conda`):
$ ~/anaconda3/bin/conda create -n new_recipe_env python=3.5
$ . ~/anaconda3/bin/activate new_recipe_env
To check that my existing “setup.py” runs I use pip to install from git, we’ll need the setup.py in the conda recipe later so we want to confirm that it works:
$ pip install git+https://github.com/ianozsvald/python_template_with_config.git # runs setup.py
# $ pip uninstall python_template_with_config # use this if you need to uninstall whilst developing
I can check that this has installed as a module using:
In : from python_template_with_config import another_module In : another_module.a_math_function() # silly function just to check that numpy is installed Out: -2.4492935982947064e-16
Now I’ll make a second conda environment to develop the recipe:
$ ~/anaconda3/bin/conda create -n new_recipe_env2 python=3.5 # vanilla environment, no numpy
$ . ~/anaconda3/bin/activate new_recipe_env2
git clone https://github.com/ianozsvald/python_template_with_config_recipe.git
cd inside, run "conda build --debug ."
The recipe (meta.yaml) will look at the git repo for python_template_with_config, pull down a copy, build using build.sh and the store a bzip2 archive locally. The build step also notes that I can upload this to Anaconda using `$ anaconda upload /home/ian/anaconda3/conda-bld/linux-64/python_template_with_config-0.1-py35_0.tar.bz2`.
A few caveats occurred whilst creating the recipe:
- You need a bld.bat, build.sh, meta.yaml, at first I created bld.sh and meta.yml (both typos) and there were no complaints…just frustration on my part – the first clue was seeing “source tree in: /home/ian/anaconda3/conda-bld/work \n number of files: 0” in the build output
- When running conda build it seems to not overwrite the version in ~/anaconda3/pkgs/ – I ended up deleting “python_template_with_config-0.1-py35_0/” and “python_template_with_config-0.1-py35_0.tar.bz2” by hand just to make sure on each build iteration – I must be missing something here, please enlighten me
- Having deleted the cached versions and fixed the typos I’d later see “number of files: 14”
- Later I added “run_tests.py” rather than “run_test.py”, I knew it wasn’t running as I’d added a “1/0” line inside run_tests.py that obviously wasn’t running (it should raise a ZeroDivisionError even if the tests did run ok). Again this was a typo on my part
- The above is tested on Linux, it ought to work on Windows but I’ve not tested it
- This meta.yaml installs from github, there’s a commented out line in there showing how to access the local source files instead
I didn’t get as far as uploading this to Anaconda to make it ‘public’ (as I don’t think that’s so useful) but I believe that final step is easy enough.
- conda recipe tutorial
- meta.yaml specification
- Notes on laying out a Python package (as a precursor to making a setup.py file)
- In my recipe in github I’ve added a set of useful links in the meta.yaml and run_tests.py
, lives in London and is a consumer of fine coffees.