Blog of Raivo Laanemets

There has been some recent discussion about the (best) practices for developing and publishing packs. For informative purposes I have decided to document my own practices and workflow here.

So far I have published 7 packs:

• arouter
• dict_schema
• docstore
• markdown
• simple_template
• sort_dict
• blog_core

All of them share the similar project structure and development workflow.

Project structure

The minimal pack contents is specified in here. I also add:

• tests directory
• README.md
• LICENSE

The tests directory contains the test code but the main reason for inclusion is that it provides additional example code. I'm not entirely sure if that has actually been a good idea as there have been ideas of running tests automatically after installation. That might not work for all of my packs.

The README file contains the project documentation, smaller examples with description and various other information. This is the most important piece of documentation the pack user will read (besides the code). This is also what the pack user first sees when browsing the pack on the SWI homepage or in a GitHub/Bitbucket-styled project/repo hosting site.

The LICENSE file is required as the pack meta-data does not have any field for the license information.

Module name prefixing

In order to avoid module name clashes, name prefixes have to be used. For example, for the simple_template pack I chose the st_ prefix and put the module files under the prolog/st directory. Subdirectory is not strictly required with packs but it made uninstall easier and avoided file name conflicts before packs were added to SWI and 3rd party libs were installed directly to the SWI libraries location.

For single-module packages I have not used a prefix as I have named the module the same as the package. It would be rather counter-intuitive and confusing to not do that. This should avoid unexpected name clashes.

README file structure

The file contains:

• Description - a paragraph/sentence about what the pack is doing.
• Quick example showing an use case.
• API overview - description of the main predicates.
• Link to generated API docs - these are displayed on SWI site as well, maybe I do not have to host these myself too?
• How to build and run tests.
• Installation (mainly useful for people browsing the project on GH or some other repo hosting).
• Changelog - lists release dates and changes in a compact format.
• Known issues - good for unstable stuff.
• Project repo location/where to send bug reports and feature requests.
• License

Build system

I expect the pack archive file to be buildable from the project repository. I generally do not expect it to be buildable from the files included in the pack archive file itself as the build process might require a specific environment or a toolset. For build/publish automation I use make and I have a fairly simple set of targets:

• test - runs tests, does not require pack to be installed
• package - creates the pack archive file
• doc - creates API docs using pldoc
• upload - uploads the pack archive file and docs to the server

The example file can be found from here. Care must be taken to not include the Makefile in the pack archive file. If you include the project Makefile in the pack, then the pack is treated as a foreign even when it contains no native code. I learned it by the hard way by having a Makefile included and seeing all the weird results when make all was run in a location where it was not expected.

Versioning

Pack versioning is quite arbitrary at the moment. I prefer semantic versioning. Michael Hendricks prefers it too and actually points that out in his projects' README files. However, version numbers are only used during publishing and you cannot easily install a specific version or set dependencies by version numbers/ranges.

Publishing workflow

1. Write code and tests.
2. Make sure tests run from the project root.
3. Write README.md containing the information above.
4. Generate API docs, upload to somewhere (unnecessary?).
5. Include API docs link in the README (unnecessary?)
6. Build pack archive file.
7. Use pack_install with the file.
8. Verify installed files.
9. If the last step fails, do changes and repeat from (6) until things work.
10. Tag version 0.0.1 in the project repo.
11. Upload the pack file somewhere.
12. Publish the pack by using the remote URL during install.

Republishing/update workflow

1. Update code and tests.
2. Regenerate API docs and reupload (unnecessary?)
3. Update README.md API and Changelog sections.
4. Pump the version number according to the changes and tag commits in the repo.
5. Rebuild the pack archive file.
6. Upload the pack file somewhere.
7. Publish the pack by using the remote URL during install.

Hacks

Install by linking

This allows to install a pack from a local location, without copying files but by symlinking to the pack project root. This enables make/0 based refresh of the pack code. This is incredibly useful when developing a pack and checking that dependant packs and projects are still working with the changes made.

Currently it possible to link manually by symlinking:

~/lib/swipl/pack/pack_name -> ~/pack_project_root

However, there is a gotcha. If you happen to run pack_install(pack_name), it wipes your project files!

Project-local packs

This approach allows to load pack code from an arbitrary location in the same way as the pack was installed by normal means. When the pack is installed into the packs/pack_name directory (relative to the current working directory) then it can be made available in the project by adding a clause to the user:file_search_path predicate (into the main file, before any use_module call that uses the pack):

:- asserta(user:file_search_path(library, 'packs/pack_name/prolog')).

And then loading a module from the pack works the same as the pack was installed into a standard location using the pack_install predicate.