How The Download Page Works
1.0 Overview
The Download page for the Fossil self-hosting repository is implemented using unversioned files. The "download.html" screen itself, and the various build products are all stored as unversioned content. The download.html page uses XMLHttpRequest() to retrieve the /juvlist webpage for a list of all unversioned files. Javascript in the download.js file (which is sourced by "download.html") then figures out which unversioned files are build products and paints appropriate icons on the displayed download page.
Except, the "Source Tarball" download products are not stored as unversioned files. They are computed on-demand by the /tarball web page.
When a new version is generated, the developers use the fossil uv edit command to make minor changes to the "download.js" file so that it knows about the new version number. Then the developers run the fossil uv add command for each build product. Finally, the fossil uv sync command is run to push all the content up to servers. All three self-hosting repositories for Fossil are updated automatically.
2.0 Details
The current text of the "download.html" and "download.js" files can be seen at:
Notice how the hyperlinks above use the "mimetype=text/plain" query parameter in order to display the file as plain text instead of the usual HTML or Javascript.
The default mimetype for "download.html" is text/html. But because the entire page is enclosed within
<div class='fossil-doc' data-title='Download Page'>...</div>
Fossil knows to add its standard header and footer information to the document, making it look just like any other page. See "embedded documentation" for further details on how <div class='fossil-doc'> this works.
With each new release, the "releases" variable in the javascript on the download.js page is edited (using "fossil uv edit download.js") to add details of the release.
When the JavaScript in the "download.js" file runs, it requests a listing of all unversioned content using the /juvlist URL. (sample /juvlist output). The content of the download page is constructed by matching unversioned files against regular expressions in the "releases" variable.
Build products need to be constructed on different machines. The precompiled binary for Linux is compiled on Linux, the precompiled binary for Windows is compiled on Windows10, and so forth. After a new release is tagged, the release manager goes around to each of the target platforms, checks out the release and compiles it, then runs fossil uv add for the build product followed by fossil uv sync to push the new build product to the various servers. This process is repeated for each build product.
When older builds are retired from the download page, the download.js page is again edited to remove the corresponding entry from the "release" variable and the edit is synced using fossil uv sync. This causes the build products to disappear from the download page immediately. But those build products are still taking up space in the unversioned content table of the server repository. To purge the obsolete build products, one or more fossil uv rm commands are run, followed by another fossil uv sync. It is important to purge obsolete build products since they take up a lot of space. At /repo-tabsize you can see that the unversioned table takes up a substantial fraction of the repository.
3.0 Security
Only users with the "y" permission are allowed to push unversioned content up to the servers. Having the ability to push check-ins (the "i" permission) is not sufficient.
On the Fossil project there are 67 people (as of 2017-03-24) who have check-in privileges. But only 3 core developers can push unversioned content and thus change the build products on the download page. Minimizing the number of people who can change the build products helps to ensure that rogue binaries do not slip onto the download page unnoticed.