Fossil

Timeline
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

About branch markdown-footnotes

The primary purpose of a footnote is to provide a reader with supplementary information in an unobtrusive way, so that the reader's attention is not distracted from the main line of a narrative. Thus a natural length of a footnote is usually a couple of sentences, maybe a paragraph. If a footnote exceeds a paragraph then it may be beneficial to rearrange a composition and incorporate that footnote as a subsection of the current (or some other) document and point there using a regular hyperlink.

There may be (at least) two ways to think of footnotes:

  1. From the viewpoint of a document's author (i.e. Markdown syntax):

    Within this point of view a footnote is defined either in the place where the corresponding numeric marker is inserted or at some other location in the document (e.g. after a paragraph or at the end of a document's source) and referenced by the corresponding label.
    The former will be called as inline while the later as referenced (or labeled).

  2. From the viewpoint of a person who reads a rendered document:

    Within this point of view either a footnote is explicitly associated with a specific text or it is the task of the reader to deduce an exact phrase to which that footnote applies. The former will be called span-bounded (or fragment-bounded) and the later as free-standing.

Hence there are four types of footnotes (suggestions for better naming are welcomed), and this branch implements all of them. Footnotes rendering is supported in all places where Markdown is supported (but see issues and limitations).

The syntax

Proposed syntax is documented along with the other markdown rules.1 It was desired to have a syntax that naturally extends conventional Markdown and which is consistent for all four types of footnotes.2

Free-standing labeled footnotes

It seems that by "footnotes in Markdown" Internet typically means referenced free-standing case. The syntax for that is more or less widespread and settled as:

  Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed doeiusmod
  tempor incididunt[^label] ut labore et dolore magna aliqua.

  [^label]: Ut enimad minim veniam, quis nostrud exercitation ullamco
     laboris nisi utaliquip ex ea commodo consequat.

The advantage of referenced footnotes is that a single footnote may be referenced multiple times and the place(s) of use may appear before the footnote's definition.

Free-standing inline footnotes

These may be more convenient for simple cases and as noted in the forum might be easier in maintenance.

For the time being, free markdown processors that support inline footnotes are unknown. The following variants were considered:

  1.  Lorem ipsum dolor sit amet(^Ut enimad minim veniam)
   
  2.  Lorem ipsum dolor sit amet[^Ut enimad minim veniam]
   
  3.  Lorem ipsum dolor sit amet^[Ut enimad minim veniam]
   
  4.  Lorem ipsum dolor sit amet^(Ut enimad minim veniam)

The first variant was chosen since it looks a bit more natural in the source form.

Span-bounded footnotes

These seems to be a pretty rare thing.
For referenced footnotes the following syntax variants were considered:

  1.  Lorem ipsum [dolor sit amet][^label]
   
  2.  Lorem ipsum [dolor sit amet](^label)
  
  3.  Lorem ipsum [dolor sit amet]^[label]
  
  4.  Lorem ipsum [dolor sit amet]^(label)

and for inline footnotes the following:

  1.  Lorem ipsum [dolor sit amet](^Ut enimad minim veniam)
  
  2.  Lorem ipsum [dolor sit amet][^Ut enimad minim veniam]
  
  3.  Lorem ipsum [dolor sit amet]^[Ut enimad minim veniam]
  
  4.  Lorem ipsum [dolor sit amet]^(Ut enimad minim veniam)

In the examples above both the scope of the footnote and its content are pretty short. That might become the typical case but the parser is capable of a longer snippets that span several lines (albeit not paragraphs).

In all cases the scope is specified inside square brackets because it is consistent with the syntax for regular links. The first variant in each category was selected as it seems consistent with the syntax for regular hyperlinks and simplifies implementation.

The corresponding text fragment of a span-bounded footnote is highlighted when a user follows footnote's back-reference or when that span is hovered over.

Styling with user-provided classes

If a footnote's text starts with a token of the special form then this token is used to derive a set of CSS classes that are added to that footnote and its references. This enables users to style elements of a particular footnote provided that the administrator provisioned and documented some special CSS classes in a custom skin. Default skin does not provide any of such special classes.

The token must start with a dot and must end with a colon; in between of these it must be a dot-separated list of words; each word may contain only ASCII alphanumeric characters and hyphens.

Linting

The numbers for misreferences, unreferenced footnotes and joined footnotes (that have several definitions with the same label) are counted.

If any of these counter is non-zero then TH1 variable $footnotes_issues_counters is set to the space separated list of corresponding integers. This simplifies reporting about issues with footnotes from within a header of a page (if such warning is provisioned in a custom skin).

Also --lint-footnotes option is added to the test-markdown-render command3. If this flag is given and footnotes in the input have issues, then the above-mentioned counters are printed to stderr and non-zero exit code is set.

Known issues and limitations

  1. There is an issue for webpages where <base href="..."> is inconsistent with the actual REQUEST_URI of a page. As of this writing preview tabs of the /wikiedit and /fileedit pages are affected. There was an attempt to fix base href but as of this writing it's not merged onto mainline. An attempt to fix it by including RQUEST_URI into the generated hyperlinks does not seem to help for these AJAXified pages.
    There is a hope to solve this issue before version 2.19 of Fossil.

  2. A footnote's text is parsed and rendered in "inline mode". This is what is needed for the usual case, but it means that block-level markup might not work inside of a footnote. If the "block-level" markup is desired then it can be done via HTML tags in the text of a footnote.

  3. Source text of an inline footnote may not contain blank lines. This limitation comes from the current architecture of the parser where paragraphs are identified before the links (and footnotes alike).
    That does not seem like a big problem in the light of the second point and the provisioned purpose of footnotes (described in the beginning).

Notes about implementation

A static integer counter is incremented upon each rendering of a Markdown document within a single process execution. This enables to generate unique IDs for footnotes and back-references within a single webpage. The counter is incremented for each invocation of markdown_to_html() in the hope for better durability of hyperlinks that point to a footnote (or a footnote's marker) within a particular post of the forum.

Counters for issues are global variables (g.ftntsIssues). The other options were considered as over-complication.

Security considerations

This implementation adds three places (above the ordinary Markdown processing) where the user input is used to construct HTML.

  1. REQUEST_URI is escaped via newly added escape_quotes() function before getting into the values of href attributes.

  2. Labels and source code of unreferenced footnotes are processed in the same way as other code blocks in Markdown source (via html_escape() function).

  3. User provided classes are constrained to ASCII alphanumeric characters and hyphens.

Thus it is believed that this feature does not introduce vulnerability for HTML injection.

Performance considerations

Footnotes processing always terminates. The amount of CPU required for the processing of typical footnotes should be on par with the rest of the Markdown processor. The worst theoretical case constitutes of a long chain of recursively nested inline footnotes. For this case the theoretical complexity is O(n2). Thus to keep CPU consumption in bounds the maximal depth of nesting that is considered is limited to 5.

See also:

Wikipedia
GitHub
StackExchange
https://daringfireball.net/2005/07/footnotes
https://www.markdownguide.org/extended-syntax/
https://support.typora.io/Markdown-Reference/#footnotes
https://michelf.ca/projects/php-markdown/extra/#footnotes
https://rephrase.net/box/word/footnotes/syntax/

Footnotes


  1. ^ If this version of Fossil supports4 footnotes then the actual syntax should be documented at /md_rules.
  2. ^ Some examples (albeit unusual) may be seen at /doc/markdown-footnotes/test/markdown-test3.md
  3. ^ A link to the corresponding check-in should also test extraction of backlinks from within footnotes.
  4. ^ These four footnotes should test the functionality of footnotes on Wiki pages.
50 most recent check-ins related to "markdown-footnotes"
2022-04-23
21:32
Extend [/md_rules#ftnts|Markdown] with footnotes support. See [/wiki?name=branch/markdown-footnotes#il|known limitations] and the corresponding [forum:/forumthread/ee1f1597e46ec07a|forum thread]. check-in: 3990518b29 user: george tags: trunk
17:23
Change signature of <code>add_inline_footnote()</code> in order to move away from returning of unreliable pointer. Amend a few comments. Fix a couple of minor issues that fuzzer complains about. Closed-Leaf check-in: 0850862e6a user: george tags: markdown-footnotes
15:56
Fix a possible heap-buffer-overflow in <code>parse_htmlblock()</code> introduced by [1e919d601f774fdb]. This is not related to footnotes but was revealed by fuzzing (case 80cbb6b185807e98a953426af7b1f802c9d13957). check-in: bc4c5b6311 user: george tags: markdown-footnotes
2022-04-22
12:49
Remove redundant <code>assert()</code> that fails for the case when the content of a span-bounded inline footnote is rendered into a void. This is a corner case that was revealed via fuzzing. check-in: cab8a586ae user: george tags: markdown-footnotes
2022-04-21
22:31
Add a test case for fragment-bounded footnote that contains markup within the corresponding text fragment. check-in: d38ec43d00 user: george tags: markdown-footnotes
21:13
Fix another use-after-realloc bug in handling of inline footnotes which was discovered during fuzzing. Also fix a few other issues revealed via fuzzer. check-in: c5456211f4 user: george tags: markdown-footnotes
13:36
Revert Makefile.in to normal builds (leave a few comments about fuzzing). check-in: 940779668f user: george tags: markdown-footnotes
13:16
Fix a use-after-free bug in handling of nested inline footnotes. The bug was discovered by fuzzing with <code>'-fsanitize=fuzzer,undefined,address -DFOSSIL_FUZZ'</code> appended to <var>TCCFLAGS</var> in Makefile.in. It's noteworthy that the <code>',undefined,address'</code> part was essential to find the bug (otherwise just 'double-free' was reported). Many thanks to Stephan for documenting the fuzzing procedures and support. check-in: 31e5df5fa2 user: george tags: markdown-footnotes
2022-04-20
14:07
Added a missing blob initializer. check-in: 7209593814 user: stephan tags: markdown-footnotes
11:48
Merged in trunk for fuzz.c changes. check-in: c9f4013530 user: stephan tags: markdown-footnotes
11:46
Correct fuzz.c to honor --fuzztype markdown and add --fuzztype wiki2 which works like its previous --fuzztype wiki behavior, sending all inputs through both the fossil-wiki and markdown translators. Added a fatal error for --fuzztype artifact, as that tester is not implemented. check-in: 8d4c479208 user: stephan tags: trunk
2022-04-19
15:25
Remove unnecessary field from the auxiliary union <code>'bitfield64_t'</code> and amend the corresponding comments. Also add comment about FOOTNOTES_WITHOUT_URI macro. check-in: cf1e96918e user: george tags: markdown-footnotes
12:35
Code style tweaks, typos, and resolved a couple footnotes-related cosmetic TODOs. No functional changes. check-in: 3a5b3d5e49 user: stephan tags: markdown-footnotes
11:41
Merged in latest trunk to simplify code review and ease potential upcoming merge to trunk. check-in: 8a4b099fb4 user: stephan tags: markdown-footnotes
2022-04-16
16:29
Minor spelling corrections. No change in functionality. check-in: 53754fff0c user: andybradford tags: trunk
2022-02-23
12:33
Count overnesting as the fourth type of the footnote-related issues and report accordingly. check-in: ae297bb671 user: george tags: markdown-footnotes
09:45
Minor refactoring. Move the definition of <code>BLOB_APPEND_LITERAL()</code> macro from <code>markdown_html.c</code> to <code>blob.c</code> so that it could be used outside of <code>markdown_html.c</code>. Also rename it to lowercase for consistency with other API. Within <code>markdown.c</code> use that newly available macro instead of <code>blob_append_string()</code>. Within <code>markdown_html.c</code> use it for footnotes-relevant code. Other invocations of <code>BLOB_APPEND_LITERAL()</code> within <code>markdown_html.c</code> are left intact (they use an alias) in order to simplify the potential merge with the trunk. check-in: c8a8d0c94c user: george tags: markdown-footnotes
08:21
Minor refactoring. Move the definition of <code>BLOB_APPEND_BLOB()</code> macro from <code>markdown_html.c</code> to <code>blob.c</code> so that it could be used outside of <code>markdown_html.c</code>. Also rename it to <code>blob_appendb()</code> for consistency with <code>blob_appendf()</code> and other API. Within <code>markdown.c</code> use that newly available macro where appropriate. Within <code>markdown_html.c</code> use it for footnotes-relevant code. Other invocations of <code>BLOB_APPEND_BLOB()</code> within <code>markdown_html.c</code> are left intact (they use an alias) in order to simplify the potential merge with the trunk. check-in: 33a681ebee user: george tags: markdown-footnotes
07:36
Fix handling of user-provided classes for unreferenced, joined and overnested footnotes. In all these cases the tokens of user-provided classes are rendered as plain-text and no special classes are added anywhere. check-in: 875472a8b0 user: george tags: markdown-footnotes
2022-02-21
05:14
Add a comment for <code>append_footnote_upc()</code>. Also substitute a variable of zero value with just "0" constant. No functional changes. check-in: ae8a3dd5aa user: george tags: markdown-footnotes
04:29
Impose a limit on the depth of nesting of inline footnotes. Also add a few test cases: for depth limiting and HTML hijacking. check-in: f4ff013ace user: george tags: markdown-footnotes
2022-02-20
23:00
If there are issues with footnotes then set TH1 variable <var>$footnotes_issues_counters</var> to a space separated list of integers that count for "misref", "unref" and "joins". This eliminates the need for JavaScript for the case when a custom skin wants to [forum:/forumpost/119b0be29a2b096b|warn about issues with footnotes] in the header of a page.<br> Also fix counting of "joins": count the number of unique labels that have multiple definitions (and not the number of such definitions). check-in: 773cef5cf7 user: george tags: markdown-footnotes
2022-02-19
01:16
Parse inline footnotes even if a renderer does not define a callback for rendering of footnote markers. This seems more correct even though the current implementation of backlink processor does define such callback as an empty function. check-in: e06c12d176 user: george tags: markdown-footnotes
01:00
Handle some corner cases more thoroughly: dismiss empty footnotes, passthrough (more carefully) user-provided classlist if the token is not followed by a blank character or if a footnote's text consists just of such token and blank characters. Also simplify a little bit a few places inside of <code>is_footnote()</code> function. check-in: fe3157803f user: george tags: markdown-footnotes
2022-02-18
01:33
Add <code>--lint-footnotes</code> option to the <code>test-markdown-render</code> command. If this flag is given and footnotes in the input have issues, then print to <var>stderr</var> the counters of "misrefs", "strays" and "split-defs" and exit with error. This should partially address a concern [forum:/forumpost/119b0be29a2b096b|raised at the forum]. check-in: 1f525713ff user: george tags: markdown-footnotes
2022-02-17
22:09
If a footnote's text starts with a token of the special form then use this token to derive a set of CSS classes that are added to that footnote and its references. This enables users to style elements of a particular footnote provided that the administrator provisioned and documented some special CSS classes in a custum skin. Default skin does not provide any of such special classes which makes this feature an "opt-in". check-in: 92516ced8b user: george tags: markdown-footnotes
00:17
Clean-up and rephrase some comments. check-in: a62c876896 user: george tags: markdown-footnotes
2022-02-16
23:08
Make parsing slightly faster and fix a comment. No changes in functionality. check-in: a36dd09d17 user: george tags: markdown-footnotes
22:11
Include <code>REQUEST_URI</code> into footnotes' hyperlinks. This should make links work even if base href (in a page's header) is not consistent with the <code>REQUEST_URI</code>. If <code>FOOTNOTES_WITHOUT_URI</code> macro is defined while compiling <code>src/markdown_html.c</code> then bare "#fragment" hyperlinks (without <code>REQUEST_URI</code>) are generated. check-in: 2c1f8f3592 user: george tags: markdown-footnotes
2022-02-14
23:32
Minor code refactoring: rename a temporary variable and utilize <code>matching_bracket_offset()</code> one more time. No changes in functionality. check-in: 5b845a0790 user: george tags: markdown-footnotes
2022-02-13
19:29
Fix parsing of "free-standing" footnotes that was (slightly) broken by the previous check-in. check-in: 23c3e0b2a7 user: george tags: markdown-footnotes
2022-02-12
20:52
If markup is ambigous between a "span-bounded" footnote and a "free-standing" footnote followed by another footnote then interpret as the later case. check-in: b363a4dbe7 user: george tags: markdown-footnotes
2022-02-11
01:26
Fix parsing of a multiline definition of labeled footnote for the case when lines end with CR+LF. check-in: ea66d15cf3 user: george tags: markdown-footnotes
2022-02-10
23:00
Clean-up and polish relevant CSS and HTML's class names. Insure visual spacing between footnotes' markers so that numbers are distinguishable when multiple footnotes in a row are used. Factor out auxiliary decorations from HTML into the default CSS, to enable customization via skins. check-in: 2b1375abad user: george tags: markdown-footnotes
2022-02-09
22:59
Handle unreferenced footnotes. If a labeled footnote is defined but there are no references to it, then add a special item at the end of footnotes. This item includes a label and the text of the strayed footnote - both rendered verbatim via <tt>html_escape()</tt>. Default skin makes such items visible and easily distinguishable. The order of such items match the order in the underlying source code. check-in: ada55cd45a user: george tags: markdown-footnotes
20:23
Cherrypicked [92221aaa192e82] and [7283ae6e120c10] on behalf of George. check-in: f902814db6 user: stephan tags: trunk
20:09
Join duplicated footnotes slightly faster. Fix a comment about auxiliary <tt>cmp_footnote_id()</tt> function. check-in: 7f6a641808 user: george tags: markdown-footnotes
19:38
Fix a misuse of an unsigned integer in the <tt>blobReallocMalloc()</tt> which can lead to redundant memory reallocations. check-in: 92221aaa19 user: george tags: markdown-footnotes
19:29
Fix a bug in the <tt>blob_reserve()</tt> function that was introduced by [1243bf39996b8a]. The <i>current</i> mainline is not affected because this function is not used anywhere. However it was causing memory corruption on the 'markdown-footnotes' branch since it was employed in [544df852b2d9a1]. check-in: 7283ae6e12 user: george tags: markdown-footnotes
2022-02-08
14:09
An attempt to fix a "double free crash" from the previous check-in. check-in: 18c9d10368 user: george tags: markdown-footnotes
14:04
If several footnotes are defined with the same label then join them into a single footnote. Text from each definition becomes an item in the list. This solution makes such situations noticable for the usual case (when this is an oversight) but also not obtrusive for the rare cases (when this is intentional). The list is provided with a special class to enable styling via skin customization.<br><b>This check-in is known to cause crash, see the forthcoming check-in.</b> check-in: 544df852b2 user: george tags: markdown-footnotes
13:39
Add <tt>const</tt> qualifier to the arguments of the <tt>blob_compare()</tt> function. check-in: 2822b63b39 user: george tags: markdown-footnotes
2022-02-06
22:53
Handle misreferences more thoroughly. Implement support of footnotes-within-footnotes with (hopefully) proper crosslinking (that's where it's getting tricky). check-in: 1787f6df11 user: george tags: markdown-footnotes
2022-02-04
23:07
Handle misreferences: a reference to undefined footnote. check-in: 28e6a9cd13 user: george tags: markdown-footnotes
19:47
Minor code refactoring. check-in: 2636e2245e user: george tags: markdown-footnotes
19:24
Automatically render a horizontal rule before the list of footnotes. If desired a particular skin can hide it using CSS selector "hr.footnotes-separator". check-in: 6807b434a9 user: george tags: markdown-footnotes
19:08
Add file [/doc/markdown-footnotes/test/markdown-test3.md|test/markdown-test3.md] that is suggested as an accumulator of footnotes-specific test cases. check-in: fe9e6ff9eb user: george tags: markdown-footnotes
17:28
For rendering a numeric footnote mark enclose HTML tag "a" inside of tag "sup" (instead of the opposite) and format anchor's id using "noteref%s-%i-%s" template (instead of "noteref-%s%i-%s"). Add highlighting when hovering over a span-bounded footnotes. check-in: fb999972e4 user: george tags: markdown-footnotes
16:54
Fix documentation so that an example of a referenced footnote definition inside of the fenced code block is not recognized as a real footnote defenition. This demonstrates a subtle gotcha and a possible work-arround of it. check-in: 7229d0f588 user: george tags: markdown-footnotes
00:37
Switch to <tt>(^...)</tt> for inline footnotes. Implement span-specific footnotes. Add [/doc/markdown-footnotes/src/markdown.md#ftnts|documentation]. check-in: cae7a5d1ca user: george tags: markdown-footnotes